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Preface 



We welcome you to the proceedings of the 5t International Conference on E- 
Commerce and Web Technology (EC- Web 2004) held in conjunction with DEXA 
2004 in Zaragoza, Spain. This conference, first held in Greenwich, United King- 
dom in 2000, now is in its fifth year and very well established. As in the four 
previous years, it served as a forum to bring together researchers from academia 
and commercial developers from industry to discuss the current state of the art 
in e-commerce and Web technology. Inspirations and new ideas emerged from 
intensive discussions during formal sessions and social events. 

Keynote addresses, research presentations and discussions during the con- 
ference helped to further develop the exchange of ideas among the researchers, 
developers and practitioners present. 

The conference attracted 103 paper submissions and almost every paper was 
reviewed by three program committee members. The program committee se- 
lected 37 papers for presentation and publication, a task which was not easy due 
to the high quality of the submitted papers. 

We would like to express our thanks to our colleagues who helped with 
putting together the technical program: the program committee members and 
external reviewers for their timely and rigorous reviews of the papers, and the 
organizing committee for their help in the administrative work and support. We 
owe special thanks to Gabriela Wagner, Mirella Koster, and Birgit Hauer for 
their helping hands concerning the administrative and organizational tasks of 
this conference. 

Finally, we would like to thank all the authors who submitted papers, authors 
who presented papers, and the participants who together made this conference 
an intellectually stimulating event through their active contributions. 

We hope that those who attended enjoyed the hospitality of Zaragoza. 
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Using Attributes to Improve Prediction Quality 
in Collaborative Filtering 



Taek-Hun Kim and Sung-Bong Yang 

Dept, of Computer Science, Yonsei University 
Seoul, 120-749, Korea 
{kimthun,yang}@cs . yonsei . ac .kr 



Abstract. To save customers’ time and efforts in searching the goods 
in the Internet, a customized recommender system is required. It is very 
important for a recommender system to predict accurately by analyzing 
customer’s preferences. A recommender system utilizes in general an in- 
formation filtering technique called collaborative filtering, which is based 
on the ratings of other customers who have similar preferences. Because 
a recommender system using collaborative filtering predicts customer’s 
preferences based only on the items without useful information on the 
attributes of each item, it may not give high quality recommendation 
consistently to the customers. In this paper we show that exploiting 
the attributes of each item improves prediction quality. We analyze the 
dataset and retrieve the preferences for the attributes because they have 
not been rated by customers explicitly. In the experiment the MovieLens 
dataset of the GroupLens Research Center has been used. The results on 
various experiments using several neighbor selection methods which are 
quite popular techniques for recommender systems show that the recom- 
mender systems using the attributes provide better prediction qualities 
than the systems without using the attribute information. Each of the 
systems using the attributes has improved the prediction quality more 
than 9%, compared with its counterpart. And the clustering-based rec- 
ommender systems using the attributes can solve the very large-scale 
dataset problem without deteriorating prediction quality. 



1 Introduction 

Nowadays, customers spend so much time and efforts in finding the best suit- 
able goods since more and more information is placed on-line. To save their 
time and efforts in searching the goods they want, a customized recommender 
system is required. A customized recommender system should predict the most 
suitable goods for customers by retrieving and analyzing their preferences. A 
recommender system utilizes in general an information filtering technique called 
collaborative filtering which is widely used for such recommender systems as 
Amazon.com and CDNow.com[l][2][3][4]. A recommender system using collabo- 
rative filtering, we call it CF, is based on the ratings of the customers who have 
similar preferences with respect to a given (test) customer. 



K. Bauknecht, M. Bichler, and B. Proll (Eds.): EC-Web 2004, LNCS 3182, pp. 1-10, 2004. 
(c) Springer- Verlag Berlin Heidelberg 2004 
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CF calculates the similarity between the test customer and each of other 
customers who have rated the items that are already rated by the test customer. 
CF uses in general the Pearson correlation coefficient for calculating the simi- 
larity, but it assumes that there must exist at least one item which has already 
been rated by both the test customer and one of the other customers. A weak 
point of the “pure” CF is that it uses all other customers including “useless” 
customers as the neighbors of the test customer. Since CF is based on the ratings 
of the neighbors who have similar preferences, it is very important to select the 
neighbors properly to improve prediction quality. Another weak point of CF is 
that it never considers customer’s preferences on the attributes of each item. 

There have been many investigations to select proper neighbors based 
on neighbor selection methods such as the /c-nearest neighbor selection, the 
threshold-based neighbor selection, and the clustering-based neighbor selection. 
They are quite popular techniques for recommender systems[l][4] [6] [7] [10]. These 
techniques then predict customer’s preferences for the items based on the results 
of the neighbors’ evaluation on the same items. 

The recommender system with the clustering-based neighbor selection 
method is known to predict with worse accuracy than the systems with other 
neighbor selection methods, but it can solve the very large-scale problem in 
recommender systems. With millions of customers and items, a recommender 
system running existing algorithms will suffer serious scalability problem. So it 
is needed a new idea that can quickly produce high prediction quality, even for 
the very large-scale problem [8] [10]. 

CF works quite well in general, because it is based on the ratings of other 
customers who have similar preferences. However, CF may not provide high 
quality recommendations for the test customer consistently, because it does not 
consider the attributes of each item and because it depends only on the ratings of 
other customers who rated the items that are already rated by the test customer. 
To improve prediction quality, CF needs reinforcements such as utilizing “useful” 
attributes of the items and using a more refined neighbor selection method. Item 
attributes mean its originality of the item it holds which differs from the others. 
In general, they can be obtained through the properties of the goods in real 
world. 

In this paper we show that exploiting the attributes of each item improves 
prediction quality. We analyze the dataset and retrieve the preferences for the 
attributes, because they have not been rated by the customers explicitly. In the 
experiment the MovieLens dataset of the GroupLens Research Center has been 
used[ll]. The dataset consists of 100,000 preferences for 1,682 movies rated by 
943 customers explicitly. 

We show the impact of utilizing the attributes information on the prediction 
qualities through various experiments. The experimental results show that the 
recommender systems using the attribute information provide better prediction 
qualities than other methods that do not utilize the attributes. Each of the 
systems using the attributes has improved the prediction quality more than 9%, 
compared with its counterpart. And besides the clustering-based CF using the 
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attributes can solve the very large-scale problem without deteriorating prediction 
quality. 

The rest of this paper is organized as follows. Section 2 describes briefly 
CF and several neighbor selection methods. Section 3 illustrates how to utilize 
the attributes of items for recommender systems in detail. In Section 4, the 
experimental results are presented. Finally, the conclusions are given in Section 5. 

2 Collaborative Filtering 

and Neighbor Selection Methods 

CF recommends items through building the profiles of the customers from their 
preferences for each item. In CF, preferences are represented generally as numeric 
values which are rated by the customers. Predicting the preference for a certain 
item that is new to the test customer is based on the ratings of other customers 
for the ‘target’ item. Therefore, it is very important to find a set of customers, 
called neighbors, with more similar preferences to the test customer for better 
prediction quality. 

In CF, Equation (1) is used to predict the preference of a customer. Note 
that in the following equation w a ,k is the Pearson correlation coefficient as in 
Equation (2) [2] [3] [4] [6] [8]. 



y X ( Tk,i ~ r k )} 




In the above equations P a ,i is the preference of customer a with respect to 
item i. ry and rF are the averages of customer a’s ratings and customer k’s 
ratings, respectively. r k ,i and r k j are customer fc’s ratings for items i and j, 
respectively, and r a j is customer a’s rating for item j. 

If customers a and k have similar ratings for an item, w a k > 0. We denote 
I w at k | to indicate how much customer a tends to agree with customer k on 
the items that both customers have already rated. In this case, customer a is 
a “positive” neighbor with respect to customer k, and vice versa. If they have 
opposite ratings for an item, then w a ,k < 0. Similarly, customer a is a “negative” 
neighbor with respect to customer k , and vice versa. In this case | w a ^ k | indicates 
how much they tend to disagree on the item that both again have already rated. 
Hence, if they don’t correlate each other, then w a ,k = 0. Note that w a .k can be 
in between -1 and 1 inclusive. 

Although CF can be regarded as a good choice for a recommender system, 
there is still much more room for improvement in prediction quality. To do so, 
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CF needs reinforcements such as utilizing “useful” attributes of the items as 
well as a more refined neighbor selection. In the rest of this section we describe 
several neighbor selection methods. 

2.1 The fc-Nearest Neighbor Selection 

The fc-nearest neighbor method selects the nearest fc neighbors who have similar 
preferences to the test customer by computing the similarities based on their 
preferences. It only uses the k neighbors who have higher correlation with the 
test customer than others, while CF suffers from considering all the customers 
in the input dataset for calculating the preference of the test customer. Thus CF 
should even consider some customers who may give bad influences on prediction 
quality. It has been shown in several investigations that the recommender system 
with the fc-nearest neighbor selection method has better quality of prediction 
than the “pure” CF[1][4][6]. 

2.2 The Threshold-Based Neighbor Selection 

The threshold-based neighbor selection method selects the neighbors who belong 
to a certain range with respect to the similarities of the preferences. Contrary 
to the fc-nearest neighbor selection method, the number of neighbors selected by 
this method varies, because it selects neighbors according to a certain threshold 
value t. 

In the recommender system with the threshold-based neighbor selection, the 
positive neighbors whose correlations to the test customer are greater than and 
equal to r are selected as the neighbors [6]. However, it is also needed that we 
include the “negative” neighbors whose correlations to the test customer are less 
than and equal to r, because they could contribute toward the better neighbor 
selection for the test customer as they provide negative “opinions” to the test 
customer. It is obvious that selecting only negative neighbors results in worse 
prediction qualities than the case in which only positive neighbors are selected, 
intuitively. 

2.3 The Clustering-Based Neighbor Selection 

The fc-means clustering method creates fc clusters each of which consists of the 
customers who have similar preferences among themselves. In this method we 
first select fc customers arbitrarily as the initial center points of the fc clusters, 
respectively. Then each customer is assigned to a cluster in such a way that the 
distance between the customer and the center of a cluster is minimized. The 
distance is calculated using the Euclidean distance, that is, a square root of 
the element-wise square of the difference between the customer and each center 
point. The Pearson correlation coefficient can be substituted for the Euclidean 
distance. 

We then calculate the mean of each cluster based on the customers who 
currently belong to the cluster. The mean is now considered as the new center of 
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Items 



Fig. 1 . The item-attributes matrix for a customer 



the cluster. After finding the new center of each cluster, we compute the distance 
between the new center and each customer as before in order to find the cluster 
to which the customer should belong. Recalculating the means and computing 
the distances are repeated until a terminating condition is met. The condition 
is in general how far each new center has moved from the previous center; that 
is, if all the new centers moved within a certain distance, we terminate the loop. 

If the clustering process is terminated, we choose the cluster with the shortest 
Euclidean distance from its center to the test customer. Finally, prediction for 
the test customer is calculated with all the customers in the chosen cluster. 



3 Using Attributes to Improve Prediction Quality 



In order to utilize the attributes of the items in the process of prediction, we 
consider a matrix of items-attributes for a customer as in Fig. 1. This figure 
shows the information of the items-attributes for customer a. In this figure r a $ 
denotes customer a’s rating for item i. Note that r a ,i £ {— , 1, 2, 3, 4, 5}, where 
indicates “no rating”. An item may have an arbitrary combination among j 
attribute values as its attribute values; Attribute (?) C {1, 2, 3, . . . , j}. Note that 
Attribute (i) 0. And € {0, 1}. 

^2{w a ,k X ( r k ,i - A(rj—))} 

Pa,i = A(r a> i ) H . , ■ ■ (3) 

2^ I W »A | 

k 



A(r a ,i ) = 



x r «} _ X 

J _ i 



N n 



, r- a = 






(4) 
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A (rj—) = — 



l x 






N, 



k.i 



M, 



k,j 



( 5 ) 



We propose Equation (3) as a new prediction formula in order to predict 
customer’s preferences. In this equation, A(r^~i) and in Equations (4) 

and (5) are the averages of customer a and fc’s attribute values, respectively. 
In each equation, N a ^ and is the number of “valid” attributes for item i 
and M a j and is the number of “valid” items that has attribute j for a 

customer, respectively. The valid attribute means • = 1 and A k i - = 1 and 
the valid item means r a y and J'k.i is not respectively. All other terms in the 
above equations are the same terms defined for the previous equations. 

There are a lot of items which have different attributes for the item for new 
prediction. Therefore, if we retrieve the attributes of the items more accurately 
and use them to the prediction process with Equation (3), then we can get more 
accurate prediction quality than the case without considering the attributes. 



4 Experimental Results 

4.1 Experiment Environment 

In order to evaluate the prediction accuracy of the proposed recommendation 
system, we used the MovieLens dataset of the GroupLens Research Center [11]. 
The dataset consists of 100,000 preferences for 1,682 movies rated by 943 cus- 
tomers explicitly. The customer preferences are represented as numeric values 
from 1 to 5 at an interval of 1, that is, 1, 2, 3, 4, and 5. A higher value means 
higher preference. In the MovieLens dataset, one of the valuable attributes of an 
item is “the genre” of a movie. There are nineteen different genres as shown in 
Table 1. 



Table 1. The genre attributes of a movie 



Unknown 


Action 


Adventure 


Animation 


Children’s 


Comedy 


Crime 


Documentary 


Drama 


Fantasy 


Film-Noir 


Horror 


Musical 


Mystery 


Romance 


Sci-Fi 


Thriller 


War 


Western 





For the experiment, we have chosen randomly 10% of customers out of all 
the customers in the dataset as the test customers. The rest of the customers are 
regarded as the training customers. For each test customer, we chose ten movies 
randomly that are actually rated by the test customer as the test movies. The 
final experimental results are averaged over the results of five different test sets 
for a statistical significance. 




Using Attributes to Improve Prediction Quality in Collaborative Filtering 



7 



4.2 Experimental Metrics 

One of the statistical prediction accuracy metrics for evaluating a recommender 
system is the mean absolute error (MAE). MAE is the mean of the errors of the 
actual customer ratings against the predicted ratings in an individual prediction 
[1] [6] [7] [8] . MAE can be computed with Equation (6). In the equation, N is the 
total number of predictions and is the error between the predicted rating and 
the actual rating for item i. The lower MAE is, the more accurate prediction 
with respect to the numerical ratings of customers we get. 

N 

i £i i 

1 E l= ^~N — • (6) 



4.3 Experimental Results 

We compared the recommendation systems using the attribute information with 
those without using them. We have implemented three recommender systems 
each of which uses different neighbor selection method. The first system is CF 
with the fc-nearest neighbor selection method ( KNCF ). The second system is 
CF with the threshold-based neighbor selection ( TNCF ). The third one is CF 
with the k- means clustering (. KMCF ). 

For TNCF, three different systems have implemented. The first system is 
TNCF with positive neighbors (T N C F _P) . The second one is TNCF with neg- 
ative neighbors ( TNCFJV ). And the last one is TNCF with both positive and 
negative neighbors (TNCF_A). 

For KMCF, we also have two different settings. One is KMCF with the 
Euclidean distance as a distance ( KMCF_E ). And the other is KMCF with the 
Pearson correlation coefficient as a distance (KMCF JO). 

The experimental results are shown in Table 2. We determined the parame- 
ters which gave us the smallest MAE through various experiments. The value of 
k for KNCF is the number of neighbors with k highest correlation. Among the 
parameters in TNCF, positive and negative values denote that the r values for 
the Pearson correlation coefficients. For example, if the positive vale is 0.2 and 
the negative value is -0.1, then we select a neighbor whose similarity is either 
smaller than and equal to -0.1 or larger than and equal to 0.2. If there is a single 
parameter, then we select neighbors whose similarities with respect to the value 
only. And the last the value of k for KMCF is the number of clusters. 

As shown in Table 2, each system has been tested both with and without 
utilizing the attributes. The results in the table show that the systems using the 
attributes outperform those without considering them. The prediction accuracy 
improvement ratio of each system using the attributes to the one without con- 
sidering them is more than 9% except for the clustering systems, KMCF_E and 
KMCF_C. 

Table 3 shows four cases of combination according to whether Equation (4) 
or (5) is applied to Equation (3) or not. If Equation (4) or (5) is not used, 
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Table 2. The experimental results 



Recommender systems 


MAE’s 


Parameters 


Attributes 


Improvement ratios 


KNCF 


0.686170 

0.761210 


fc=130 

fc=120 


Used 
Not used 


9.86% 


TNCF.P 


0.689162 

0.760716 


o o 
to to 


Used 
Not used 


9.41% 


TNCF_N 


0.745659 

0.838030 


T=-1.0 

T=-1.0 


Used 
Not used 


11.02% 


TNCF_A 


0.681760 

0.754619 


mmmmm 


Used 
Not used 


9.66% 


KMCF.E 


0.744384 

0.774412 


k= 2 
k= 2 


Used 
Not used 


3.88% 


KMCF.C 


0.729218 

0.762834 


k= 2 
k= 2 


Used 
Not used 


4.41% 



Table 3. The usage of the attributes in prediction 



Cases 


A 


B 


C 


D 


Used equations 


(4) and (5) 


(4) only 


(5) only 


none 



then the item corresponded to Equation (1) is substituted. Table 4 shows the 
experimental results on these cases. 

The experimental results in Table 4 show us that the prediction with case A 
that the attributes are applied to both the test customer and the neighbors, pro- 
vides the best prediction accuracy in KNCF and TNCF. And in these methods 
the prediction with case B also has as good prediction quality as the predic- 
tion with case A. On the other hand KMCF provides the best prediction when 
we use the case B in both KMCF_E and KMCF_C. The prediction accuracy 
improvement ratio of KMCF with case B to the case D is more than 9%. 

In the results we found the TNCF_A (case A) is the best prediction quality 
among others. And we can see the fact that the prediction with case A through 
C provides more prediction accuracy than the one with case D which never 
considers the attributes of an item. And the last the KMCF_C (case B) provides 
as good as TNCF_A in prediction quality. This fact means that the clustering- 
based CF can solve the very large-scale problem without deteriorating prediction 
quality. 

5 Conclusions 

It is very crucial for a recommender system to have a capability of making ac- 
curate prediction by retrieving and analyzing of customer’s preferences. Collab- 
orative filtering is widely used for recommender systems. Hence various efforts 
to overcome its drawbacks have been made to improve prediction quality. 

It is important to select neighbors properly in order to make up the weak 
points in collaborative filtering and to improve prediction quality. In this paper 
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Table 4. The experimental results(for four cases) 



Methods 


MAE 


Parameters 


Types 


KNCF 


0.686170 


fc=130 


A 




0.692809 


fc=200 


B 




0.767625 


fc=120 


C 




0.761210 


fc=120 


D 


TNCF.P 


0.689162 


<N 

© 

II 


A 




0.696775 


s 

II 

o 

to 


B 




0.767223 


r=0.2 


C 




0.760716 


t=0.2 


D 


TNCF.N 


0.745659 


T— -1.0 


A 




0.745667 


T— -1.0 


B 




0.838022 


T— -1.0 


C 




0.838030 


T— -1.0 


D 


TNCF.A 


0.681760 


r=(0.2,-0.1) 


A 




0.687949 


r=(0.2,-0.1) 


B 




0.761572 


r=(0.2,-0.1) 


C 




0.754619 


r=(0.2,-0.1) 


D 


KMCF.E 


0.744384 


CM 

II 


A 




0.704240 


k = 2 


B 




0.811596 


k = 2 


C 




0.774412 


k = 2 


D 


KMCF.C 


0.729218 


CM 

II 


A 




0.692414 


k = 2 


B 




0.796347 


CM 

II 


C 




0.762834 


k = 2 


D 



we showed that the recommender systems that exploit the attributes of each 
item indeed improve prediction qualities. The experimental results show the 
recommender systems have improved the prediction qualities more than 9%. 
And besides the clustering-based CF using the attributes can solve the large- 
scale problem without deteriorating prediction quality. 
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Abstract. The numerous web sites existing nowadays make available more in- 
formation than a user can manage. Thus, an essential requirement of current 
web applications is to provide users with instruments for personalized selective 
retrieval of web information. In this paper, a procedure for making personalized 
recommendations is proposed. The method is based on building a predictive 
model from an association model of Web data. It uses a set of association rules 
generated by a data mining algorithm that discovers knowledge in an incre- 
mental way. These rules provide models with relevant patterns that minimize 
the recommendation errors. 



1 Introduction 

A critical issue of modem Web applications is the incorporation of mechanisms for 
personalized selective retrieval of Web information. In the e-commerce environment, 
this is a way of increasing customer satisfaction and taking positions in the competi- 
tive market of the electronic business activities. Traditional, not electronic, companies 
usually improve their competitiveness by means of business intelligence strategies 
supported by techniques like data mining. Data mining algorithms find consumers’ 
profiles and purchase patterns in the corporate databases that can be used for effective 
marketing and, in general, for business decision making. In the field of the e- 
commerce these procedures can also be applied but they have been extended to deal 
with problems of the web systems, such as information overload [3], The incorpora- 
tion of efficient personalization methods in these applications contributes to avoiding 
this problem and therefore, increases business benefits. The aim is to take advantage 
of the information obtained from customers’ accesses in order to be able to make 
recommendations for a given customer about products that could be among his pref- 
erences. 

Two types of error can appear in poor recommender systems: false negatives, 
which are products that are not recommended, though the customer would like them, 
and false positives, which are products that are recommended, though the customer 
does not like them [4], False positive errors are less accepted by the clients than the 
false negative ones, therefore they are the most influential in the customer churn phe- 
nomenon. Data mining techniques contribute to the development of efficient personal- 
ized recommender systems by finding customers characteristics that increase the 
probability of making right recommendations. Data mining problems can be resolved 
by employing supervised and unsupervised algorithms. Unsupervised algorithms are 
usually used in knowledge discovery modeling, but they can be successfully used for 
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predictive tasks in classification problems. The process of applying data mining tech- 
niques on web data is known as web mining. There are three categories: Web content 
mining, Web structure mining and Web usage mining. Web content mining is the 
process of discovering useful information on Web contents (text, image, audio, video, 
etc.). Web structure mining is based on the hyperlinks’ structure of Web pages. Web 
usage mining aims to find interesting patterns, such as user profiles, from user 
logs [2], 

Web mining tasks can be carried out automatically by means of software agents. 
The use of agents in web applications is widely extended, mainly in search engines 
and e-commerce environments, where they play diverse roles. The dynamic search of 
relevant information is one of the major roles [10] [2] [9]. Their autonomy, learning 
capability and the possibility of working in cooperation with other agents are suitable 
properties for their use in the personalization of web systems by means of data mining 
techniques. We have used them in an e-commerce intermediary site architecture [5], 
and we are now focused in its personalization aspect. 

In this work, we present a web mining method for making recommendations based 
on the predictive use of an association rules’ model that relates user and product at- 
tributes. Most of the existing association algorithms have the drawbacks that they 
discover too many patterns which are either obvious or irrelevant and, sometimes, 
contradictions between rules appear. We propose a refinement method in order to 
obtain stronger rules that reinforce the relation between items. An architecture of 
intelligent agents is suggested for systems implementing the method; one of the 
agents is in charge of doing data mining tasks, generating the associative models used 
in the recommendations. As products, customers and preferences change with time, 
models must be frequently modified. The cooperation between the system agents 
allows the automatic updating and the fitting of the recommendations to the new 
models. Our proposal should provide recommender systems with more relevant pat- 
terns that minimize the recommendation errors. The main contributions of this work 
are discussed in the section of results and also in the conclusions. 

The next sections of the paper are outlined as follows. Section 2 summarizes re- 
search related to recommender systems. In section 3.1 the association analysis 
foundations and the proposed refinement algorithm are described. The proposed 
recommender system is presented in section 3.2. Section 4 includes the experimental 
study and results. Finally, we present the conclusions and future work. 



2 Related Work 

Recommender systems are used to increase sales by offering a selection of products 
or services a consumer is likely to be interested in. There are two main categories of 
recommendation methods: collaborative filtering and a content-based approach [8]. 
The first technique was originally based on nearest neighbor algorithms, which pre- 
dict product preferences for a user, based on the opinions of other users. The opinions 
can be obtained explicitly from the users as a rating score or by using some implicit 
measures from purchase records such as timing logs [18], In the content based ap- 
proach text documents are recommended by comparing their contents and user pro- 
files [8], The lack of mechanisms to manage Web objects such as motion pictures, 
images or music constitutes the main weakness of this approach in the e-commerce 
application area. Besides, it is very difficult to handle the big number of attributes 
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obtained from the product contents. Collaborative filtering also has limitations in the 
e-commerce environment. Rating schemes can only be applied to homogeneous do- 
main information. Besides, sparsity and scalability are serious weaknesses which 
would lead to poor recommendations [4]. Sparsity is due to the number of ratings 
needed for prediction is greater than the number of the ratings obtained because usu- 
ally collaborative filtering requires user explicit expression of personal preferences 
for products. The second limitation is related to performance problems in the search 
for neighbors. 

In the last years many recommender systems based on the last approach have been 
developed. The GroupLens research system [7], Ringo [18] and Video Recommender 
[6] are three examples based on this approach. The usual technique used in these 
systems is based on correlation coefficients. The method requires user ratings about 
different recommendable objects. Correlation coefficients showing similarities be- 
tween users are computed from the ratings. Then, recommendations based on these 
coefficients can be made. The procedure presents the sparsity problem and the first- 
rater problem that takes place when new products are introduced [7], 

There are two approaches for collaborative filtering, memory-based ( user-based ) 
and model-based ( item-based ) algorithms. Memory-based algorithms, also known as 
nearest-neighbor methods, were the earliest used [18]. They treat all user items by 
means of statistical techniques in order to find users with similar preferences 
(neighbors). The prediction of preferences (recommendation) for the active user is 
based on the neighborhood features. A weighted average of the product ratings of the 
nearest neighbors is taken for this purpose. The advantage of these algorithms is the 
quick incorporation of the most recent information, but they have the inconvenience 
that the search for neighbors in large databases is slow [19]. 

Data mining technologies, such as Bayesian networks, clustering and association 
rules, have also been applied to recommender systems. Model-based collaborative 
filtering algorithms use these methods in the development of a model of user ratings. 
This recent approach was introduced to reduce the sparsity problem and to get better 
recommender systems. 

The Bayesian network analysis is a technique that formulates a probabilistic model 
for collaborative filtering problem. The underlying structure used for classification is 
a decision tree representing user information. The predictive model is built off-line by 
a machine learning process and used after to do recommendations to the active users. 
The process is fast and simple and this is very suitable for systems in which consumer 
preferences change slowly with respect to the time needed to build the model [19]. 

In a recent work [3], the authors propose the use of methods from both categories 
in order to avoid the commented problems. The support vector machine (SVM) mem- 
ory-based technique is used for content-based recommendations, and the latent class 
model (LCM), that is a model-based approach, is used for collaborative recommenda- 
tion. 

Rule-based approaches have also been applied to overcome problems that 
personalized systems have [8]. The data should be processed before generating the 
rules. In [4] a recommendation methodology that combines both data mining 
techniques is proposed. First a decision tree induction technique is used in the 
selection of target customers. Later, association rules are generated and used for 
discovering associations between products. 
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Clustering techniques identify groups of users who appear to have similar prefer- 
ences. Predictions are based on the user participation degree in the clusters. These 
techniques are not very accurate and they can be useful in a preliminary exploration 
of the data. 

A graphical technique that has yielded better results than nearest neighbors is hort- 
ing [21]. Nodes in the graph represent users and edges between nodes indicate degree 
of similarity between users. Prediction is produced by walking the graph to nearby 
nodes and combining the opinions of the nearby users. 

The algorithm of nearest neighbors has also been applied in combination with data 
mining techniques. Lee et al. [8] create a user profile that is effective within a specific 
domain by using a nearest neighborhood-based method. They expand the traditional 
method to find profiles valid over all the domains. For each user, neighbors’ transac- 
tion information is used to generate Web object association rules. 

The main advantage of Web mining methods is that they allow avoiding the prob- 
lems associated with traditional collaborative filtering techniques [12]. 

Our proposal is a different model-based approach that deals with the case of new 
users in which an initial user profile is not available. We apply an association rule 
algorithm in order to find initial patterns. Rating and users information is used in the 
rule generation procedure. We use other users’ attributes to refine the rules and obtain 
specific rules that consider the particularities of each user. The algorithm drives the 
refinement procedure for obtaining the most useful rules for the desired objective 
(recommendation of products). 



3 Making Personalized Recommendations 

The recommendation procedure is based on searching the user profile in order to 
personalize the recommendations for him/her. The profiles are given by a set of re- 
fined association rules that relate information about products preferences and user 
attributes. The procedure for obtaining them is explained below. 



3.1 Generating and Refining Association Rules 

Mining association rules from data is mostly used for finding purchase patterns in 
commercial environments. Algorithms for discovering rules, such as “Apriori”, the 
first to be introduced [1], are not complex; however, they usually generate a large 
number of rules, even restricting them with high values of support and confidence. In 
this work, we present a refinement algorithm that reduces the number of association 
rules generated and produces the best rules considering their purpose, which is, in our 
case, product recommendation. The refinement process is based on the concept of 
unexpectedness. The foundations of association rules and unexpectedness [16] are 
introduced below. 

Consider the set of N transactions D=[Tj ,T 2 , ,T N } over the relation schema 

{ij , i 2 , ,i m } consisting on a set of discrete attributes. Also, let an atomic condi- 

tion be a proposition of the form value ; < attribute < value , for ordered attributes and 
attribute = value for unordered attributes where value, value ; and value 2 belong to the 
set of distinct values taken by attribute in D. In [16] rules are defined as extended 
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association rules of the form X -A A, where X is the conjunction of atomic condi- 
tions (an itemset) and A is an atomic condition. The Rules’ strength and applicability 
is given by the factors: Confidence or predictability. A rule has confidence c if c% of 
the transactions in D that contain X also contain A. A rule is said to hold on a dataset 
D if the confidence of the rule is greater than a user-specified threshold value chosen 
to be any value greater than 0.5. Support or prevalence . The rule has support s in D if 
s% of the transactions in D contain both X and A. Expected predictability. This is the 
frequency of occurrence of the item A. So the difference between expected predict- 
ability and predictability is a measure of the change in predictive power due to the 
presence of X [3], 

In [15] the initial rules are a set of beliefs that come from domain knowledge. A 
rule A — > B is defined to be unexpected with respect to the rule X -A Y on the data- 
base D if the following conditions hold: 

■ B and Y logically contradict each other (B AND Y |= FALSE); 

■ A AND X holds on a “large” subset of tuples in D; 

■ The rule A, X -A B holds. 

For example, an initial rule X -A Y is that women like comedy movies (women -A 
comedy). The rule scientist -A documentary (A — > B) is unexpected with respect to 
the initial rule if the above conditions hold. 

The concept of unexpectedness, is the basis of the proposed algorithm. The proce- 
dure also uses the best attributes for classification in order to find the more suitable 
rules for the purpose of recommendation. As a result, a small set of appropriate and 
highly confident rules that relate user data and product attributes is produced. It con- 
stitutes a very simple model for recommending products. 

In this work, we are using a discovery-knowledge technique for solving a classifi- 
cation problem. We aim to build a model that relates some user attributes with prod- 
uct attributes in order to make recommendations of products. Thus, either the product 
or the category of products can be considered as the class label. The label attribute is 
the target of prediction in classification problems. Importance of columns is a tech- 
nique that determines how important various attributes (columns) are in discriminat- 
ing the different values of the label attribute [11]. A measure called purity (a number 
from 0 to 100) informs about how well the columns discriminate the classes (different 
values of the label attribute). It is based on the amount of information (entropy) that 
the column (attribute) provides. We use this technique for finding the best attributes 
used in the refinement algorithm. 

The algorithm requires generating initial beliefs and unexpected patterns. In [15] 
the beliefs can either be obtained from the decision maker or induced from the data 
using machine learning methods. In our case, those beliefs were generated from the 
entire data-base by an association rule algorithm [11]. In an earlier work [14] we 
found that the use of refined rules for prediction in the projects’ management area 
improved the results obtained with supervised techniques [13]. The recommender 
procedure proposed in this paper follows the approach of using good attributes for 
clasification in a rules’ refinement algorithm wich works with Web usage data. We 
start with a set of beliefs which relate items. Then we search for unexpected patterns 
that could help us to increase the confidence or to solve ambiguities or inconsistencies 
between the rules representing the beliefs. 
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The refinement process used for software size estimation [14] has been modified in 
order to adapt it to Web data, which are mostly discrete. The steps to be taken are 
described below: 

1. Obtain the best attributes for classification and create the sequence: seqA = <A k >, 
k = 1 ...t (t: number of attributes). The attributes in the sequence are ordered from 
greater to lesser purity. 

2. The values of the attribute A k are represented as { V /f ; ) , / = 1 ...m (nr. number of 
different values). 

3. Set k = 1 and establish the minimal confidence c ■ and minimal support s , . 

4. Generate initial beliefs with confidence c > c min and support s > s min . 

5. Select beliefs with confidence near c min or with conflicts between each other: 

Let X ( . — > Y ; and X. — > Y. be two beliefs, R ( and respectively. There is a con- 
flict between R. and R if X, = X, and Y, ->= Y,. 

' J 1 J 1 J 

6. With the selected beliefs create the rule set setR = {R ; }, i = 1 ...n (n: number of 
selected beliefs) 

7. For all beliefs R ; e setR do: 

7.1. Use the values { ,} of the attribute A, for generating unexpected pattern 

fulfilling conditions of unexpectedness and confidence > c min . The form of 
the patterns is: ; > B. 

7.2. Refine the beliefs by searching for rules R’ like: 

X,-,V w -> B 

Y,. 

7.3. Let setR’ be the set of refined rules, then the beliefs refined in step 7.2 should 
be added to it: 

setR’ = setR’ u { R’ u ], u = 1 ...f(f: number of refined rules obtained in the 
iteration i). 

8. Set k = k+ 1 and setR = setR’. 

9. Repeat steps 7 and 8 until no more unexpected patterns can be found. 

A positive aspect of this approach is the incremental knowledge discovery by using 
good attributes for classification progressively. Other methods for association rules 
mining take all available attributes. Thus, a great set of rules is generated, which is 
pruned later without considering classification criteria. Our algorithm simplifies the 
process of patterns selection and generates the best rules for the predictive purpose of 
recommending products. 

3.2 Recommender System 

The main objective of this proposal is to provide a simple procedure, which contribute 
to avoid the problems of the recommender systems, mentioned previously. 

Firstly, we build the associative model that will be used for making the recommen- 
dations. In order to reduce the number of association rules generated and to obtain 
rules applicable to a wide range of customers, a list of the best rated products is gen- 
erated. A minimum threshold value of the ratings was established for selecting the 
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products. Afterward, the association rules are generated from the list. The recommen- 
dations are based on the patterns obtained. The initial rules are refined by means of an 
algorithm described in the section 3.1 in order to obtain strong patterns that contribute 
to avoid the false positive errors. The refined rules, which relate product attributes 
with user attributes and preferences, represent customer profiles. They are used for 
building a predictive model that finds interesting products for a given customer with a 
specific profile. Recommendation for new users can be made by checking their char- 
acteristics. The procedure allows the recommendation of new products whose charac- 
teristics belong to the user profile. New products with new characteristics are always 
recommended. This is the way to deal with the first-rater problem. The information 
about new products and new users provides feedback to the system. New association 
models are built periodically from the new information. 




Fig. 1 . UML activity diagram for the recommendation scenario 



The system architecture contains three main agents that take charge of interacting 
with the user, managing the information and generating the recommender models. 
Their cooperation in the recommendation scenario is shown in figure 1. The data 
mining agent applies the proposed algorithm for generating association rules’ model, 
which is composed by rules at different levels of refinement. This agent uses the in- 
formation provided by the data management agent periodically. The recommenda- 
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tion agent transforms the associative model in a predictive model. It receives the 
requests from the users, takes their profiles and uses the predictive model for making 
the personalized recommendations. The most refined rules are used in the first in- 
stance. If the user has a profile coincident with the antecedent part of the rule, the 
preferential recommendation is constituted by products that satisfy the consequent 
part. If there are not rules that match the profile, previous level of refinement is used, 
and so forth. These levels are also used for secondary recommendations. The problem 
of finding more than one rule matching the profile is solved by taking the more confi- 
dent rule for preferential recommendation. The data management agent collects and 
manages the storage of the information about new user preferences and new products. 
It is connected with the data mining agent to provide the data that are used periodi- 
cally in the generation of new models. 



4 Experimental Study 

MovieLens data sets, collected by the GroupLens Research Project at the University 
of Minnesota, were used in the experimental study. The database contains user demo- 
graphic information and user rating about movies, collected through the MovieLens 
Web site (movielens.umn.edu) during a seven-month period. All movies belong to 18 
different genres. User ratings were recorded on a numeric five point scale. Users who 
had less than 20 ratings or did not have complete demographic information were re- 
moved from the data set. It consists of 100,000 ratings (1-5) from 943 users on 1682 
movies (each user has rated at least 20 movies). 

Initially, the rating information was used to extract the best valued movies. 
Association rules were produced by taking the records with a rate value greater than 
2. Before generating rules, we obtain the best attributes for discriminating the 
GENRE label. The attributes found by means of the Mineset tool [11] were 
OCCUPATION, AGE, and GENDER. These attributes were used in the refinement of 
the association rules. Thus, the initial beliefs in our case are rules that relate GENRE- 
TIME STAMP” with the user's occupation. Time stamp is the time that the user spent 
in a product; it is another way of rating the products. Refined rules use the next “best 
attribute”, AGE, combined with the first, OCCUPATION. Figures 2 and 3 are graphi- 
cal representations provided by mineset tool of first and refined rules respectively on a 
grid landscape with left-hand side (LHS) items on one axis (genre-time stamp), and 
right-hand side (RHS) items on the other (user attributes). Attributes of a rule (LHS 
— > RHS) are displayed at the junction of its LHS and RHS item. The display includes 
bars, disk and colors whose meaning is given in the graph. 

We have specified a minimum predictability threshold of 30%. Rules generator 
does not report rules in which the predictability (confidence) is less than the expected 
predictability, that is, the result of dividing predictability by expected predictability 
(pred_div_expect) should be greater than one. Good rules are those with high values 
of pred_div_expect. The colors and the height of the bars in graphs (figures 2 and 3) 
show that these values increase in the refined rules. After the refinement process a 
reduced number of rules with high confidence have been obtained. Therefore, an 
enhancement of the recommendations based in these rules is produced. 

The algorithm produces the most useful rules for the recommendation process, due 
to the fact that it uses the best attributes for classification, that is, the most important 
attributes in discriminating the different values of the class attribute, which is the 
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Fig. 2. Initial beliefs 



Fig. 3. Refined beliefs 



target of the recommendations. Other rule refinement methods do not consider this 
issue, thus they reduce the number of rules, but they do not obtain the most suitable 
rules for the desired objectives. 

The simplicity of the models obtained with the refined rules lead to their efficient 
application for recommendation. On the other hand, these models are built off-line, a 
fact which implies an enhancing of the recommender procedure efficiency in terms of 
computer response time. 

With respect to the advantages of using association instead of classification, the 
first is the lower number of attributes required and the second, the greater efficiency 
of the association algorithms. Clustering is another technique used habitually for 
finding neighbors, but it also presents computer time problems previously discussed. 



5 Conclusions 

In this paper, an agent based methodology for recommendation is proposed. The 
methodology deals with the case of making recommendations for new users and with 
the firs-rated problem. The recommender procedure is based on a predictive model 
built from an associative model that contains association rules between attributes that 
can be obtained from a database containing Web usage information. Recommenda- 
tions provided by the predictive model are founded on strong patterns that contribute 
to reduce the false positive errors. Besides, there are not performance problems at 
recommendation time due to these models contain reduced number of highly confi- 
dent rules and they are built off-line. Time spent in building the associative models is 
also short because association algorithms are, in general, more efficient than the clas- 
sification ones. On the other hand, de refinement algorithm proposed selects the best 
rules for recommendation by using the best attributes for classification. 
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Abstract. An Internet advertising system proposed here clusters Web site users 
with similar preferences into numerous segments through Web usage mining. It 
utilizes fuzzy rules which express user segments’ surfing patterns and appropriate 
Web ads. It selects proper Web ads by fuzzy inference, stores them in recommen- 
dation sets database, and forwards them to the target user. To verify the effective- 
ness of the system, changes in click-through-ratio scores per e-newspaper section 
are observed. 



1 Introduction 

Rapid development of the Internet and Web technologies enables customers to have 
more choices than ever before about what to buy, where to buy, and how to buy. The 
emerging electronic commerce has also changed many aspects of existing business and 
business process. At this time, successful companies provide a bundle of customized 
services acceptable, which satisfy the customers’ needs, to gain a competitive advan- 
tage. 

As the range of the Internet techniques available to online advertisers expands, Web 
advertising attracts public attention as a new communication channel. Among others, 
banner ad continues to dominate spending in online advertising and the market for that 
is steadily growing [1], Online advertisement will become the most important revenue 
for many companies on the Web in the near future. 

Conventional advertising, however, is passive, targeted to mass audiences, and has 
suffered poor responses from customers. To raise the effectiveness of banner ads, Web 
sites need to be able to put the right message to the right customer at the right time, and 
to provide personalized advertising messages to every desirable customer. Therefore 
their online marketing solutions should possess the functionality to identify a customer, 
predict and understand his or her preferences and interests, choose appropriate ads, and 
deliver them in a personalized format directly to him or her during his or her online 
session [11]. 

In this study, an adaptive and personalized system for the Internet advertising will 
be introduced. It provides personalized ads to users whenever they visit an e-newspaper 
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Web site, which enables an online advertiser to practice one-to-one marketing. As a re- 
sult, the system improves users’ satisfaction and their responses to ads. In doing so, the 
Web ad system mines Web server logs containing users’ Web navigation patterns with 
machine learning techniques, identities their current tastes and interests, and decides on 
proper ads through fuzzy reasoning which will be forwarded to users. 

2 Internet Advertisement 

Advertising on the Web refers to advertising that delivers electronic information ser- 
vices to users. Until now, several literatures on online advertisement focus on using the 
Web for supporting commerce in terms of electronic marketing, digital publishing and 
dissemination. They analyze online ads’ effectiveness from the marketing perspective 
[10]. Langheinrich et al. [7] classify online advertisement approaches into four cat- 
egories: untargeted, editorial, targeted, and personalized. Personalized advertisement 
is regarded as a next generation advertisement approach and typically uses machine 
learning methods, such as neural networks, to allow personalized ad selection and rec- 
ommendation based on the browsing and interaction history of a particular user as well 
as other demographic information. 

Currently available personalization techniques comprise decision rule-based filter- 
ing, content-based filtering, collaborative filtering, and non-intrusive personalization 
[9]. Decision rule-based filtering asks users a series of questions to obtain user demo- 
graphics or static profiles, then lets marketing experts manually specify rules based on 
them, and delivers the appropriate items (Web ads) to a particular user based on the 
rules. However, it is not particularly useful, since it is difficult to obtain valuable rules 
from marketing experts and to validate the effectiveness of the extracted rules. 

Content-based filtering recommends items being similar to what a user has liked 
previously. Collaborative filtering selects items based on the opinions of other cus- 
tomers with similar past preferences. However, traditional recommendation techniques, 
including content-based or collaborative filtering, have some limitations, such as re- 
liance on subject user ratings and static profiles, or the inability to capture richer se- 
mantic relationships among Web objects and to scale to a large number of items. 

To overcome these shortcomings, non-intrusive personalization attempts to incor- 
porate Web usage mining techniques. Web usage mining uses data mining algorithms 
to automatically discover and extract patterns from Web usage data and predict user 
behavior while users interact with the Web. Recommendation based on Web usage min- 
ing has several advantages over traditional techniques. It can dynamically develop user 
profiles from user patterns while reducing the need to explicitly obtain subjective user 
ratings or registration-based personal preferences. Therefore the recommendation sys- 
tem’s performance does not degrade over time. 

In the e-commerce environment, analyzing such information embedded in click- 
stream data residing in Web logs is critical to improve the effectiveness and perfor- 
mance of Web marketing for online service providers. Recently there have been many 
researches on Web server log analysis from both industry and academia. Some of these 
efforts have shown how Web usage mining techniques can be used to characterize and 
model Web site access patterns and how well they are useful to recommendation in 
electronic commerce scenarios [5]. 
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3 Personalized Advertising System 

The overall architecture of adaptive Web ad personalization system comprises two com- 
ponents: online and offline. Fig. 1 depicts the online architecture showing the system’s 
essential functionality. 




Fig. 1 . Online components of the ad recommender system. It always monitors users' feedbacks 
to determine the timing of new fuzzy rules to generate 



The online components consist of interface agent, matching agent, and HTTP 
server. The interface agent provides an interface between the HTTP server and its users 
for user interaction. The Web server helps identify a user and tracks the active user 
session as the interface agent makes HTTP requests. The matching agent looks up a 
recommendation sets database to extract a set of recommended ads for a target user. 
It then sends the ads to the client browser through the interface agent. Clickthrough 
ratio (CTR) score, which is the percentage of times that viewers of a Web page click 
on a given banner ad, measures the effectiveness of the recommended ads, and further 
determines the timing of new fuzzy rules to generate. 

The offline is divided into four parts, as shown in Fig. 2: customer clustering, fuzzy 
inference, Web ad clustering, and Web ad segment selection. Customer clustering uses 
a self-organizing map (SOM), a neural clustering method, to divide Web site users 
into numerous groups with similar surfing preferences through mining Web server logs. 
Browsing patterns and recommendation sets for a user segment form fuzzy rules used 
in fuzzy inference. Fuzzy rules are mainly obtained from the previous recommendation 
and effect histories. 

Fuzzy inference receives each user’s fuzzified page views by news section from the 
Users database which stores each user’s Web site navigation history. It draws conclu- 
sions about recommended types of ads, which are fed into Web ad segment selection. 
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Fig. 2. Offline components comprise four functions: customer clustering, fuzzy inference, Web 
ad clustering, and Web ad segment selection 



Web ad clustering uses another SOM to divide Web ads into groups with similar ad 
features. Web ad segment selection chooses appropriate Web ad segments by calculat- 
ing Hamming distance measures and then Web ads by calculating Euclidean distance 
measures. The system stores the selected Web ads in the recommendation sets database 
for site users, which will be forwarded to a target user. 

3.1 Segmentation of Web Site Users and Web Ads 

To maximize the effectiveness of Web ads through recommending the right ads to the 
right users, service providers should determine Web site users’ habits and interests first, 
which can be discovered by mining their Web page navigation patterns (Web usage 
mining) [4, 6]. 

When extracting users’ access histories, on which the mining algorithms can be run, 
several data preprocessing issues - cleaning data, identifying unique users, sessions, and 
transactions - have to be addressed. Data cleaning eliminates irrelevant items including 
all entries with filename suffixes which indicate graphic hies in server logs. It leaves log 
entries indicating e-news pages. Individual users and their sessions can be identified by 
various preprocessing methods [2, 12], 

Once users and their access histories have been identified, segmenting users follows. 
It breaks users into peer groups with similar Web page navigation behaviors. Behaviors 
among the groups, however, differ significantly. Segmentation by traversal history uses 
a segmentation variable on the basis of how many times a user visits each section. 

Fig. 3 shows a process of segmenting users and fuzzifying segments’ characteris- 
tics. For analytical convenience, this study classifies electronic newspaper’s sections 
into seven categories, such as POL (politics), ECO (economy), SOC (society), WOR 
(world), CUL (culture and life), SCI (science and technology), and SPO (sports). These 





An Intelligent System for Personalized Advertising on the Internet 



25 






(b) 



Customer segments and 
their characteristics 
represented by fuzzy sets 






POL 


ECO 


SOC 


IaJOR 


CUL 


SCI 


SPO 


155.230.50. 41 


17 8 


15 6 


91 


31 


71 


66 


146 


155.230.51. 134 


163 


276 


165 


54 


10 4 


123 


357 


155.230.51. 185 


19 0 


222 


172 


56 


12 4 


149 


481 


155.230. 192.93 


119 


160 


99 


32 


70 


57 


158 


155.230.231. 46 


263 


230 


155 


84 


147 


181 


370 



Fig. 3. (a) A process for segmenting users and fuzzifying segments' characteristics, (b) Sample 
users" browsing counts per news section 



sections are later used as antecedents of fuzzy rules. The number of times each user 
has visited each section during the analysis period is counted from the Web log, and 
is fed into the SOM which builds user behavior models. In training SOM, managerial 
convenience decides on nine as the number of output units. 

Fig. 4(a) shows fuzzy partitions of sections SOC and SCI. Fig. 4(b) summarizes 
seven dominant segments derived from a three-by-three SOM, and their average access 
counts expressed by fuzzy numbers. Because users’ characteristics are generally fuzzy 
and can not be specified in crisp values, fuzzy numbers are suitable for expressing users’ 
navigation histories. Therefore four triangular fuzzy numbers such as S (small), MS 
(medium small), MB (medium big), and B (big), or three ones such as S, M (medium), 
and B are used. 

To build an experimental Web ads database for the Web ad system, I chose sample 
ads randomly from a major e-newspaper provider in Korea; ten ads under each sec- 
tion. After examining these ads, I extracted six categorical features as follows: MAG 
(newspaper and magazine-related ads), ECO (business and financial ads), COM (com- 
puter and telecommunication-related ads), SPO (sports-related ads), ENT (culture and 
entertainment-related ads), and RAN (the others). Note that each ad has fuzzy charac- 
teristics which can be represented as the combination of these six features. 

Membership degrees ranging from zero to one are appropriately assigned to each 
ad’s features. SOM then clusters Web ads with six features (i.e,, six input dimensions) 
into numerous groups with similar ad characteristics. A cluster centroid represents av- 
erage membership degree for each feature. Each of the six features has triangular fuzzy 
number which is used as consequents in fuzzy rules executed only to the degree that 
antecedents are true. 
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Fig. 4. (a) Fuzzy partitions of the society, and science and technology sections, (b) Dominant user 
segments and their characteristics expressed in fuzzy numbers 



3.2 Fuzzy Rules Generation and Fuzzy Inference 

Fig. 5 shows how to generate fuzzy rules and describes sample rules. A fuzzy rule has 
a form of IF-THEN in which the left-hand side is a condition part and a conjunction 
of triangular fuzzy numbers for access counts by section, and the right-hand side is an 
action part and a conjunction of triangular fuzzy numbers for ad features. 

The advertising system utilizes its previous recommendation and response histories 
of segmented users to make fuzzy rules for them. It is based on which ads have been 
recommended to each peer group and have received good responses. Ad experts may 
refine the fuzzy rules with domain knowledge of which ads are suitable to each user 
segment. 

Once a fuzzy rule base has been established, a fuzzy inference engine receives fuzzi- 
fied user access counts per section as an input vector value. Fuzzy inference is a math- 
ematical process to calculate the fuzzy output probability from the probability values 
of the input functions. It processes the information through the fuzzy rule base and 
produces a fuzzy set for each ad feature, which is converted to a real number through 
defuzzification. 

Mamdani’s popular MIN-MAX reasoning strategy is used as a fuzzy inference 
method [3] [ 8] . When an access count vector is given as (x\, X2, £ 3 , X4, X5, xq, Xr) = 
(POL, ECO, SOC, WOR, CUL, SCI, SPO), the probability of fuzzy output function 
can be calculated as follows (1): 

W k = M A 1 {x 1 ) A lia 2 (x 2 ) A ha 3 (x 3 ) A ha±{x 4 ) A ha 5 {x 5 ) A / x a 6 {x 6 ) A ha 7 {x 7 ) 

Mc(*)= r vVf AncM (1) 

i = 1 
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Fig. 5. Fuzzy rules generation, (a) Sample fuzzy rules, (b) A process generating fuzzy rules 



where fic(z) is a fuzzy set for an ad’s z th categorical feature, is the mem- 

bership degree for the i th news section, and x, is the i th element in an access count 
vector. 

Because seven antecedents exist in each rule, seven membership degrees are ob- 
tained. Each fuzzy rule’s fitness value is then determined by a MIN (A) operation on 
these membership degrees. After applying the fitness values to the consequents of the 
rules, combining multiple outputs through a MAX (V) operation completes the fuzzy 
inference process. 



3.3 Web Ads Recommendation and Continuous Monitoring 



Defuzzification is to determine a crisp correspondence in a set of real numbers from the 
fuzzy sets derived from fuzzy inference per each ad’s feature [3], The central gravity 
(Zdefuzzified) is chosen as a defuzzification method and is calculated by using the follow- 
ing equation (2): 



Zj defuzzified 



f fi c (z)zdz 
f nc{z)dz 



( 2 ) 



After a set of defuzzified real numbers are obtained, the ad system calculates a set of 
Hamming distance between a set of defuzzified numbers and centroids of ad segments. 
A Hamming distance is calculated by using the following equation (3): 



n 

d(A, B) = ^2 \nA{xi) - fi B (xi)\ 

xleix 



(3) 
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where X is categorical features of each ad, )j_a is defuzzified numbers made from fuzzy 
inference, ^ is centroid of each ad segment, and d( A. B) is the Hamming distance. 

For example, assuming that a set of defuzzified numbers are (MAG, ECO, COM, 
SPO, ENT, RAN) = (0.64, 0.21, 0.16, 0.35, 0.2, 0.19), a Hamming distance to a cluster 
centroid, (MAG, ECO, COM, SPO, ENT, RAN) = (0.0, 0.0, 0.0, 1.0, 0.5, 1.0), reaches 
2.77. The closer the Hamming distance, the more similar preference the ad segment 
has with regard to a target user. The member ads in the closest ad segment are the 
candidates for recommendation to the user. Euclidean distance measures between the 
cluster centroid and the ads belonging to the cluster are, in sequence, calculated and 
stored in the recommendation sets database for future recommendation. 

Whenever a user visits the Web site, the ad system looks up the recommendation 
sets database to find the suitable ads and display them on the user’s browser. However, 
because customers’ interests and needs continuously change, the system must capture 
this change. When average CTR goes down a set threshold, it starts deriving new user 
profiles and new fuzzy rules for new recommendation. 



4 Experiment and Evaluation 

Generally, several metrics used to measure the effectiveness of Web ads include the 
CTR. In most Web ads, a 2% CTR would be considered successful. Especially as ads 
become familiar to users, the CTR is well below 1%. 

In order to verify the effect of the proposed system on the increase in CTR scores 
per section of the e-newspaper, an experimental architecture for measuring the Web ad 
system’s performance consists of multiple Web servers, which operate together and deal 
with actual users of the e-newspaper provider. When a user requests e-news services, a 




Fig. 6. (a) The CTR scores of the Web ad system, (b) Changes in total average CTR scores over 
time 
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virtual server receives the request and redirects it to any free-traffic actual Web server. 
Users do not know which server they connect. If every server is busy, the virtual server, 
then, redirects the request to the randomly selected actual server. Actual servers are 
divided into two groups for the experiment: the treatment group (Web servers with the 
ad system) and the control group (Web servers without the ad system). The treatment 
group and control group hold the same e-news articles, but only the treatment group 
shows the recommended ads to users. 

Fig. 6(a) illustrates the average CTR scores per section for the treatment and control 
groups after a 80-day experiment. Several sections including Economy, Culture and 
Life, and Sports show a clear improvement in the average CTR after adoption of the 
Web ad system, while recommending ads to users frequently looking at Political, Social, 
World, or Science and Technology section seems less effective. 

Fig. 6(b) illustrates changes in total average CTR scores and a trend line during the 
experiment (Equation for the trend line is y — —2 E — 6x 2 + 0.0001a: + 0.0396 with 
R 2 = 0.6644). The CTR scores fluctuate during the analysis period, but the general 
pattern for users to view Web ads shows increase in the score at first and decrease 
afterward. If a threshold is set to 0.02 of the CTR, the system advises segmenting new 
users and creating new fuzzy rules at the 76 th day after starting the experiment. 

5 Conclusion and Future Research 

An adaptive and personalized Web ad system is proposed to raise the effectiveness of 
banner ads on the Internet. It consists of online and offline components: online has 
an interface agent, matching agent, and HTTP server; offline has customer clustering, 
fuzzy inference, Web ad clustering, and Web ad segment selection functionalities. 

The work presented here can be extended in several ways for future research. First, 
clustering analysis used to segment user’s access patterns can be extended to include 
richer information. The current clustering analysis uses an access count value (fre- 
quency value) as a base variable for user segmentation. Utilizing RFM (recency, fre- 
quency, and monetary) values altogether can be a way to improve clustering results, 
since they encompass the entire user’s behavior. 

Second, deriving and maintaining accurate fuzzy rule sets is an issue of a fuzzy sys- 
tem. The larger the number of sets, the more accurately a fuzzy system reflects the true 
relationship among variables. However, it increases complexity, decreases robustness 
of the system, and makes it more difficult to modify the system. 
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Abstract. This paper presents a new technology for supporting flexible 
query management in recommender systems. It is aimed at guiding a 
user in refining her query when it fails to return any item. It allows the 
user to understand the culprit of the failure and to decide what is the 
best compromise to chose. The method uses the notion of hierarchical 
abstraction among a set of features, and tries to relax first the constraint 
on the feature with lowest abstraction, hence with the lightest revision of 
the original user needs. We have introduced this methodology in a travel 
recommender system as a query refinement tool used to pass the returned 
items by the query to a case-based ranking algorithm, before showing 
the query results to the user. We discuss the results of the empirical 
evaluation which shows that the method, even if incomplete, is powerful 
enough to assist the users most of the time. 

1 Introduction 

Business to consumer web sites have quickly proliferated and nowadays almost 
every kind of product or service can be bought on-line. In order to improve 
the effectiveness of user decision making support, recommender systems have 
been introduced in some domains. For instance, it is very popular the books 
recommender system included in amazon.com or the movie recommender sys- 
tem MovieLens [1]. In a previous paper we have introduced Trip@dvice, a travel 
planning recommendation methodology that integrates interactive query man- 
agement and case-based reasoning (CBR) [2]. In this methodology, product re- 
search is supported by an interactive query management subsystems that coop- 
erates with the user and provides information about the cause of query failure 
and its possible remedies. The goal is to help the user to autonomously decide 
what is the best compromise when not all his wants and needs can be satisfied. 
CBR supports the modelling of the lruman/computer interaction as a case and is 
exploited to extract, from previously recorded recommendation sessions, useful 
information that enable to intelligently sort the results of a user’s query. 

In this paper we focus on the Trip@dvice relaxation algorithm and its evalua- 
tion in practical usage. Interactive query management, and in particular dealing 
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with failing queries has been addressed in a number of researches in the area of 
cooperative database [3-5]. Clru et al. suggested to explore an abstraction hier- 
archy among values of a feature/ attribue, and to use that knowledge for moving 
along the hierarchy for relaxation or tightening the result set [3] . Godfrey exten- 
sively studied the cause of failure of boolean queries, and proposed an algorithm 
for query relaxation [5]. 

Our new query relaxation algorithm extends that presented in [6], introduc- 
ing the notion of abstraction-hierarchy. This is a relationship among features 
describing the product and not, as in [3], a partition of a feature domain (group- 
ing of values). Moreover our approach is not limited to boolean constraints, as 
that discussed in [5], as we address the problem of relaxing range constraints 
on numeric attributes. In principle our goal is to relax the minimum number of 
constraints for a given query, such that a non void result is returned. But, finding 
such relaxed query is in general an NP-hard problem [5], hence we focussed on 
a simpler, incomplete but practically feasible and effective approach. 

In this paper we first present the motivation for using interactive query man- 
agement (IQM) in recommender systems and then we describe the architecture 
of the IQM module implemented in Trip@dvice. Then we present the empirical 
evaluation of IQM, that was conducted by exploiting log data derived from the 
evaluation a recommender system based on Trip@dvice [2] . The results show the 
method is powerful enough to find at lease one successful relaxed query most of 
the time. 



2 Interactive Query Management 

In previous researches on knowledge-based recommender systems the problem of 
dealing with the failure of an over-constrained query was typically addressed by 
exploiting similarity-based retrieval [7,8]. In Trip@dvice [2], as well as in other 
proposals [9], it is argued that the system should not autonomously determine 
the best attainable approximate match, as in a similarity-based retrieval, but 
should actively support the user and let him understand what is the best relax- 
ation/compromise. Hence in Trip@dvice when a query fails to return any item a 
dedicated interactive query management component (IQM in Figure 1) suggests 
some refinement(s) in terms of query constraints to discard or change. 




Fig. 1 . Intelligent Mediator architecture. 
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A user will typically interacts with IQM through a Graphical User Interface, 
submitting a query and getting as reply a reasonable set of items. If this can- 
not be achieved then IQM suggests to the user some refinements to the failing 
query. Two types of failures are considered: the query returns either too many 
results or no result at all. When a query returns an empty set, IQM tries to find 
some relaxed alternative queries that change or remove the minimum number of 
constraints in the failing query and would make the subquery to produce some 
results. IQM reports these alternative options to the user so that he can decide 
what is the best relaxation (from his point of view) and opt for this with the cer- 
tainty that the query will now return some result. Conversely, in case too many 
items would be returned, a set of candidate features are suggested as those that 
should be constrained in addition to those already included in the original query 
(this last topic is not discussed here, see [6]). 

The query language supported in Trip@dvice is simple but expressive enough 
to cope with standard web forms. Let X = \\ff =1 be an it em space (e.g., a set 
of hotels). We shall refer to a feature of X , as /* with its associated domain JQ 
for all -i, 1 < i < n. A query, say q is obtained by the conjunction of atomic 
constraints; q = ci A • • • A c m (or simply q = {c i, • • • , c m }), where m <n and each 
Cfc (1 < k < m) constrains the feature fi k . Each constraint is defined as follows 

{ fi k = true if fi k is boolean 

fik = v if fik is nominal (1) 

fik € [l, u ] if fi k is numeric 

where v,l,u € Xi k . 

Note that the query language we consider here does not support inquiries 
across catalogues, because we assume that a catalogue is a custom view that is 
possibly built over a set of joint tables. 

We now focus on the Relax module in Figure 1. We shall refer to a relaxed 
version of a query by calling it simply a subquery , and a successful subquery if it 
returns a non-empty result set. 

Example 1. Letg : {[2 < category < 3], [parking = true], [30 < price < 50]} be a 
query, then q' -.{[category > 3], [price < 35]} is a subquery of q, where the constraint 
on parking is relaxed. 

The relaxation method makes use of the notion of feature abstraction hierar- 
chy, i.e., a hierarchy among some features of an item space. For example, assume 
the accommodation item space contains the category (i.e., l-star,2-star,...), price 
and parking features. Then we say that the feature category is more abstract than 
the price because the knowledge of the price greatly reduces the uncertainty over 
the category (i.e the conditional entropy of the category given the price is low 
[10]). The idea is to use such relationships in the relaxation process to sort a set 
of related constraints and start relaxation on the constrained-feature with lowest 
abstraction level. If this relaxation does not produce any result, then this con- 
straint is removed and the constraint on the feature with next-lower abstraction 
is tried. 

More formally, we denote with FAH = {F 1 , . . . , F k } a collection of feature 
abstraction hierarchies, where each F l = (/},... , ff . ) is an abstraction hierarchy, 
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i.e., an ordered list of features in X (comparable features). We mean that f ‘ is 
more abstract than f j iff j < l , and there are no common features in two different 
F z s, i.e. features in different F l are not comparable. 

Figure 2 depicts the algorithm used in Trip@dvice to find relaxing sub-queries 
of a failing query 1 . The constraints in the query q = {ci, . . . , c m } are partitioned 
in the set CL = {cl 1, . . . , cl m '}, where each cl t is a ordered list of constraints of 
q, such that the features in cl t belongs only to one F i (line 1). If the constrained 
feature in any constraint c does not belong to any abstraction hierarchy, then a 
singleton list, containing only c, is built. So for instance, let q = {ci, 02,03} be 
the query as in Example 1, and let the feature abstraction hierarchy be F AH = 
{(category, price)}, then CL = {(01,03), (02)} is the partition of q according to 
this F AH. The proposed algorithm outputs at most two subqueries, one for each 
element in CL. The first subquery is obtained by relaxing a constraint in (ci, 03) 
the second by relaxing 02- To find the first subquery, the algorithm first checks if 
the relaxation of 03 (price) produces a successful subquery, otherwise it removes 
C 3 and relaxes also C\. 

More precisely, the loop at line 3 tries to relax the set f related constraints 
(i.e., cl £ CL) while keeping the rest of the constraints in q unchanged. The 
related constraints in cl are relaxed starting from the one on the feature with 
lowest abstraction by the for loop at line 6. ch is a list containing all the modified 
versions of the current constraint, and it is initialized with c at line 8. The 
variable relax is a flag that indicates whether a wider or shorter range should be 
tried for a numerical constraints. It is initially set to true. 



q = {ci, . . . , c m }: a query to be relaxed. 



RelaxQuery(g) 

1 CL < — Partition constraints in q according 

to the defined FAH ; 

2 QL < — 0; % the list of subqueries 

3 for each cl £ CL do 

4 q <- q; 

5 suggest * — 0;% a subquery suggestion 

6 for j — \cl\ to 1 do 

7 c * — get j th constraint from cl 

8 ch < — {c} % constraint history 

9 relax « — true 

10 do 

11 q' <- q! - {c} 

12 c< — ModifyRange(c, ch, relax) 

13 while ( Analyse (</, c, ch, relax, suggest) ) 

14 if suggest 7 ^ 0 then 

15 QL « — QL U suggest ; 

16 j <- 0 ; % no need to go to higher abstr. 

17 end:if 

18 endrfor 

19 end:for 

20 return QL 



ModifyRange(c, relax, ch) 

1 c < — c; 

2 if c is numeric then 

3 if relax then 

4 c / < — IncreaseRange(c); 

5 else 

6 c « — DecreaseRange(c, ch); 

7 end:if 

8 return c 



Analyse(q , c, ch, relax, suggest ) 

1 ret Value * — false; 

2 if ( c ^ null and 

length(ch) < maxj,ength ) then 

3 if c is numeric then 

4 ret Value < — true; 

5 q' «— q' U {c}; 

6 end:if 

7 n < — Count(q / ); 

8 if (n = 0 and relax = false) then 

9 ret Value < — false; 

10 else if (n = 0) then 

11 ch < — ch U {c}; 

12 relax < — true; 

13 else if rr > cx then 

14 relax < — false; 

15 if n > 0 then 

16 suggest * — {(q , n)}; 

17 end:if 

18 return retValue 



Fig. 2. Relaxation algorithms. 
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Variables are in italic and literals are in sans serif 
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Then the do-while loop at lines 10-13 tries different ranges for a numerical 
constraint, or it iterates only once for a non- numerical constraint. The Modi- 
fyRange function modifies only the numerical constraints. It returns a modified 
version of the input (numeric) constraint depending on the relax flag. The re- 
turned constraint contains either a wider or shorter range than the one specified 
in c depending on the value of relax. It returns null if increasing (decreasing) a 
range would not increase (decrease) the result size of the subquery. For instance, 
if the current range specifies the whole range of values in the catalogue for the 
constrained feature, then increasing the range would not improve the result size 
of current subquery. 

Let c : [l < fi < u] be current range constraint. The IncreaseRange function 
relaxes c to [I — <5J < fi < [m + <5], where 

^ (O.l(w-Z) if u > l , 2 . 

( 0.1 ('Umax limin') H 11 = l 

and t>min,Vmax are the minimum and maximum values of feature fi in the cata- 
logue, respectively. 

Example 2. (Example 1 cont.) Assume q returns an empty set. If we call Relax- 
Query to find all the relaxed subqueries of q, the IncreaseRange will be called in 
turn to find a new wider range for price, and it would return [28 < price < 52] . 

The DecreaseRange will find a range between that range that caused an 
empty result set and the current one that makes the subquery to return too 
many results. Let [Zi,«i], and [Z 2 , ^ 2 ] be two such ranges. Using this method 
to shorten [ ], it would return a new constraint specifying the range [, l',u '] 
where V = (Z 2 — li)/2 and v! = (112 — Ui)/2. 

Example 3. (Example 2 cont.) Assume trying the wider range [28, 52] for the price 
returns too many items by the corresponding subquery q' (i.e., Count(</) > a). 
The next call to the ModifyRange will call the DecreaseRange in order to reduce 
the current range, and it will return the constraint [29 < price < 51]. 

The Analyse procedure determines whether or not the maximum number of 
attempts to find a suitable constraint for a numeric feature has been reached. If 
not, it examines the result size (n) of the current subquery. If the result size is 0, 
it sets the relax flag to true to indicate a wider range should be tried, otherwise 
if the current range has made the subquery to produce a large result set, then 
the relax is set to false to indicate a shorter range has to be tried. If at any stage 
the subquery produces some results, then the method creates a new suggestion 
(i.e, suggest ) which is a set containing a pair made of the subquery and its result 
size (line 16, in the Analyse). 

After the do-while loop (lines 10-13), if there is any suggestion, then there 
is no need to relax the constraint on feature with higher abstraction, and hence 
the loop at line 6 terminates and the suggestion is added to the return set ( QL ). 

The algorithm terminates when all the lists cl £ CL of q are tried. Obviously, 
the running time of RelaxQuery is 0(\q\) where \q\ is the number of constraints 
in the query q , since each constraint is considered a constant number of times. 
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It is worth to note that the RelaxQuery returns at most the same number 
of successful subqueries as there are elements in the partition set of the query 
q , where each subquery represents a compromise (i.e., relaxation of a subset of 
constraints in the query) the user has to make in order to receive some results. 
Hence, it is up to the user to judge which subquery is the best one according 
to her preferences and constraints. For instance, if the RelaxQuery returns two 
subqueries, one that relaxes the constraints on price and the category, and an- 
other that relaxes the parking feature, then the user can choose either view more 
expensive hotels or to miss the parking. 

3 Evaluation 

We developed a prototype, based on Trip@dvice [2], and empirically evaluated 
that with real users. In fact two variants of the same system ( Nut-King 2 ) were 
built. One supports intelligent query management (IQM) and intelligent ranking 
based on cases ( NutKing + ), and the other without those functions (Nut-King-). In 
fact NutKing-h used a previous version of the relaxation algorithm, that did not 
use the notion of feature-abstraction hierarchy [6]. The subjects had randomly 
been assigned to one of the two variants. 

Table 1 shows some objective measures relative to the IQM. It includes the 
average values and standard deviations of the measures taken for each recom- 
mendation session. Values marked with * (**) means a significant difference at 
the 0.1 (0.05) probability level, according to an unpaired t-test. Relaxation was 
suggested by the system 6.3 times per session, and this means that approxi- 
mately 50% of the queries had a “no result” failure. The user accepted one of 
the proposed subqueries 2.8 times, i.e. almost 45% of the time. We consider this 
a good result, taking into account the user behavior that is often erratic and not 
always focussed in solving the task. 

Table 1 . Comparison of NutKing+ and NutKing-. 



Objective Measure 


NutKing- 


NutKing+ 


queries issued by the user 


20.1±19.2 


13.4±9.3* 


number of features in a query 


4.7±1.2 


4.4±1.1 


results size per query 


42.0±61.2 




system suggested query relaxations 


n.a. 


6.3±3.6 


# of times the user accepted query relaxations 


n.a 


2.8±2.1 



We now provide a more detailed description of the queries issued to each 
of the five catalogues used in the experiment and of the performance of the 
RelaxQuery algorithm. In particular we shall show how often RelaxQuery can 
effectively assist users. 

Five catalogues/collections are considered in NutKing: accommodations, cul- 
tural attractions, events, locations, and sport activities. We do not consider here 
the cultural attractions catalog since it has only three features and relaxation of 
1-constraint in a failing culture-query has always produced some results. 

http:/ /itr. itc.it 
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We mined the log file of users’ interactions in the empirical evaluation, and 
extracted all the users’ queries submitted to different catalogues in NutKingi. 
We grouped queries according to their type (i.e. the catalogue they queried) 
and the number of constraints they contained. Let Qi denote the set of queries 
submitted to a catalogue where each query contains j constraints. The log also 
contained the result size of each query, which enabled us to determine the set 
of failing queries, i.e., those that had a void result set. Let FQi C Qi denote 
the set of such failing queries. Then we ran (again) the RelaxQuery algorithm 
on each query q £ FQi to compare the subset of failing queries, say SQi that 
the RelaxQuery could find a successful subquery, with and without the notion 
of feature-abstraction-hierarchy (FAH) (Table 2). The relaxation algorithm that 
does not exploit FAH is described in [6]. In total using F AH the proposed 
algorithm was able to find a successful subquery 83% of the cases (232/279), 
whereas the same algorithm not using FAH was successful 61% (172/279). It 
is worth noting that, this is obtained by loosing a property of the previous 
method, i.e., the capability to find all the successful subqueries that relax 1-single 
constraint. In other words there are cases in which using FAH two constraints 
in a hierarchy are relaxed even when only one is necessary. This happen when 
relaxing the more abstract and still keeping the lees abstract does give some 
results. But, in practice, using FAH we can find a successful subquery in more 
cases. 



Table 2. Queries submitted to the catalogues. 



Catalogue 


SELl 


MaMI 


5E3I 


E SQ 3 FAH 


Accommodation 




112 


73 


94 


Location 




64 


29 


50 


Event 




57 


39 


52 


Sport 




49 


31 


46 


Total 




279 


172 


232 



The figures 3(a)-(d) compare the size of the sets Q\ FQ j , and SQ j for the 
location, accommodation, event, and sport catalogues, respectively, when FAH 
is used. 

This shows that RelaxQuery can always find a successful subquery when 
the user query contains up to three constraints. This number is even higher for 
the queries submitted to event and lodging catalogues. In particular, consider- 
ing the accommodation catalogue, we observe that more than half of queries 
failed to return any item, and the RelaxQuery could find a subquery 83% of the 
time(Table 2). We also see the method performs worse for the location-queries. 
One reason for this is location has more features than the others, and as the 
number of constraints in a query increases, the likelihood increases that a failing 
query cannot return any result even by removing one constraint. In other words, 
the more constraints a query has, the more likely it is to have a situation that 3 
constraints are inconsistent such that no item can satisfy those constraints and 
we have to remove at least 2 constraints in order to receive some results. This is 
the case for many queries of the location catalogue with 8 constraints or more. 
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# y' Cy-islrunUi ir Thy Queries 



# or C cost-amts in The Queries 



(a) Location 



(b) Lodging 




# of Constra nts in he Queries 



(c) Event 




* or Coost-aints in The Queues 



(d) Sport 



Fig. 3. Behavior of the RclaxQuery algorithm for different catalogues. 



Finally, we want to illustrate how difficult it would be for a user to refine au- 
tonomously her failing query. We note that we did not mined the log data coming 
from NutKing-, to measure if and how the user relaxed the failing queries (with- 
out system assistance). The discussion here is hypothetical, and is aimed at un- 
derstanding, in the worse case, the number of query change attempts (constraint 
removals, that are needed to receive some results discarding the minimum num- 
ber of constraints. Let q = {ci, . . . , c m j be a query, and assume the RelaxQuery 
method can find k successful subqueries. The maximum number of attempts the 
user needs in order to find a successful change to her failing that relaxes only 
one constraint is m — k + 1. In fact, if a user query receives zero results, then 
in a second attempt, the user must remove one constraint, and submit the new 
query. If this again fails he must put back the removed constraint and discard 
another constraint, and submit the new query. He must proceed until receives 
some results, which could be, in the worse case at the (m + k — l) th attempt. In 
fact, this number will be even larger if q contains some range constraint, because 
different ranges should be tried. In practice this is not feasible for a user. 

4 Related Work 

The research on failing queries goes back to the work of Kaplan [11], where 
he described a system that supported a cooperative query answering to queries 
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expressed in natural language. He argued that it is more informative to let the 
user understand the cause of failure, and provide answers that partially satisfy 
the query, than just reporting a void answer like the empty result set. 

Value abstraction was introduced in the CoBase system [3] , where an abstrac- 
tion hierarchy among feature values is built, and the relaxation operators use it 
to move up and down the hierarchy. In our approach, a hierarchy is a relationship 
between features (e.g., country^county^city), derived from domain knowledge, 
and is exploited to find successful sub-queries of a failing query. While building 
a values abstraction hierarchy is quite complex and costly in CoBase, in our 
approach the feature hierarchy can be achieved with minimal effort. 

Gaasterland et al. described in [4] a logic-based approach to develop a coop- 
erative answering system that accepts natural language queries, and can explain 
the cause of any failing query. Godfrey has extensively investigated the problem 
of identifying the cause of a failing boolean-query [5]. He has derived an algo- 
rithm (called ISHMAEL) to find successful maximal subquery (called XSS), i.e. , 
not contained in any other successful subquery. He shows that finding only one 
XSS can be done in linear time proportional to the number of constraints in q, 
but finding all XSS is intractable. The major difference between RelaxQuery and 
ISHMAEL relates to the search mode. While the latter does a depth-first-search 
to find an XSS subquery, the former takes a breath-first-search to find a sub- 
query that relaxes minimum number of constraints, hence must be incomplete 
to avoid the combinatorial explosion of search states. 

McSherry [9] has approached the relaxation problem differently by looking 
at the products that do not satisfy the query. For each product in the catalogue 
the set of attributes (called compromise set) that do not satisfy the query is 
computed, and a case representative (i.e., with highest similarity to the query) 
is put in a set called retrieval set. The problem with this approach is the retrieval 
set may contain up to 2^ elements. 
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Fig. 4. Steps needed to find a relaxation. 
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5 Conclusions and Future Work 

In this paper we presented a linear time incomplete algorithm for query relax- 
ation. If it finds a solution this is maximal with respect to the number of con- 
straints kept in the successful subquery found. Thus, since a query represents 
user’s preferences, the algorithm relaxes as few as possible user’s preferences. 
This algorithm is simple enough to be easily integrated into any eCommerce ap- 
plication. An empirical evaluation of the algorithm was presented, and showed 
the algorithm is powerful enough to recover from a failure situation most of the 
time. 

In the future we plan to introduce user modelling principles that would enable 
to assign weights to user’s preferences such that the preferences with lowest 
weight will be relaxed first. Moreover, we are extending RelaxQuery to cope 
with the general case, i.e., when more than one constraint must be relaxed to 
find a successful subquery. 
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Abstract. Most e-commerce Web sites dynamically generate their con- 
tents through a three-tier server architecture composed of a Web server, 
an application server, and a database server. In such an architecture, the 
database server easily becomes a bottleneck to the overall performance. 
In this paper, we propose WDBAccel, a high-performance database server 
accelerator that significantly improves the throughput of the database 
processing, and thus that of the overall Web site. WDBAccel elimi- 
nates costly, complex query processing needed to obtain query results 
by reusing previous query results for subsequent queries. This differen- 
tiates WDBAccel from other database cache systems, which replicate 
a database into multiple conventional DBMS’s and distribute queries 
among them. We evaluate the performance of WDBAccel by using the 
queries of the TPC-W benchmark. The measurement results show that 
WDBAccel outperforms DBMS-based cache systems by up to an order 
of magnitude. 



1 Introduction 

With the explosive growth of the Internet, numerous value-generating services 
are provided through WWW. In most e-commerce Web sites, those services 
are usually deployed on a three-tier server architecture, which consists of Web 
servers, application servers, and database servers. Web contents are dynamically 
generated upon a request through such system components. Due to its low scala- 
bility and complexity of query processing, a database server is a major bottleneck 
to the overall site performance. 

We propose a high-performance database server accelerator, called WDBAc- 
cel, which significantly improves the throughput of database processing in multi- 
tier Web sites (see Figure 1). The main approach of WDBAccel is to cache results 
of frequently-issued queries and reuse these results for incoming queries. Upon 
a query hit, the query result is immediately served from the cache. WDBAccel 
keeps cached results consistent to an origin database by invalidating staled re- 
sults. In addition, WDBAccel is designed to use main memory as the primary 
storage, minimizing disk operations. Thus, WDBAccel can improve the perfor- 
mance of database processing more than an order of magnitude. 

WDBAccel differs from other existing DB cache system [6,11,9,1] in that 
the primary purpose of WDBAccel is to accelerate the database processing. On 

K. Bauknecht, M. Bichler, and B. Proll (Eds.): EC-Web 2004, LNCS 3182, pp. 41-50, 2004. 

(c) Springer- Verlag Berlin Heidelberg 2004 
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Fig. 1 . WDBAccel deployment in an e-commerce Web site 



the contrary, the primary purpose of other cache systems is to distribute the 
load of database processing into multiple cache nodes. Those systems replicate 
a database into multiple nodes and distribute queries among them. In serving 
queries, they rely on the underlying DBMS, which executes costly query pro- 
cessing to obtain a query result. Thus, the performance of DBMS-based cache 
systems is limited by their underlying DBMS. 

Our WDBAccel design incorporates three policies that effectively utilize the 
limited space of main memory. First, WDBAccel employs the derived matching. 
Even when the cache does not store the identical query result, the result for a 
query can be derived from one or more previously stored queries. In many Web- 
based applications, selection regions of queries tend to overlap each other. Thus, 
WDBAccel performs the derived matching by containment checking among se- 
lection regions of stored query results. Second, WDBAccel removes the storage 
redundancy of the tuples belonging to two or more query results. In many cases, 
query results contain identical tuples. Therefore, WDBAccel eliminates such a 
storage redundancy by storing query results in the unit of tuples. Third, a cache 
replacement policy in WDBAccel evaluates storage cost in a different way from 
other Web caching systems. In many systems, the policy considers the size of 
cached data items. However, it is not appropriate in WDBAccel due to the tu- 
ples shared among multiple query results. WDBAccel considers both the size of 
query results and shared tuples. 

WDBAccel provides several advantages to database-driven Web sites. The 
most competitive advantage is that it drastically improves the throughput of the 
Web sites. Second, WDBAccel reduces the total cost of ownership. WDBAccel 
is a light-weight system optimized in caching and serving query results. Thus, it 
can be deployed even on lower-end H/W system while achieving a high level of 
performance. Third, WDBAccel can be easily deployed as a middle-tier solution 
between the Web application server and the database server. By supporting 
the standard interfaces like JDBC or ODBC, WDBAccel does not require Web 
applications to be modified. Forth, the high-performance nature of WDBAccel 
reduces the total number of cache nodes managed by an administrator, reducing 
administration cost. 

This paper is organized as follows. In section 2, we describe the architecture of 
WDBAccel. In section 3, we explain technical details including query matching, 
cache storage, and cache replacement. In section 4, we evaluate and analyze the 
performance of WDBAccel. In section 5, we describe related works and compare 
them to our system. Finally in section 6, we present conclusions. 

2 System Architecture 

WDBAccel is a Web database server accelerator which processes queries deliv- 
ered from the front-side Web application servers (WAS’s) on behalf of database 
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servers. It is designed as a middle-tier system which can be deployed between 
WAS’s and database servers without extensive modification to either system as 
other middle-tier cache systems [6,11,9]. To deploy WDBAccel on an existing 
e-commerce service system, it is only required to change an existing database 
driver to WDB Accel’s driver at the WAS. 

WDBAccel is a query result caching system. As mentioned, it stores the re- 
sults of previous queries and then serves incoming queries delivered from WAS. 
The query result caching is extremely useful when a workload is read-dominant. 
The workload of most e-commerce applications is read-dominant. In e-commerce 
sites, visitors spend the most time finding and reading some information, e.g., 
product catalogs, news, articles, etc. Update interactions such as ordering prod- 
ucts are relatively very infrequent. For example, the TPC-W benchmark [12] 1 
specifies that the portion of read queries, in an average case, is 80% of the entire 
workload. Thus, we can expect that the query result caching will show a high 
level of performance in many e-commerce sites. 

Figure 2 shows the overall architecture of WDBAccel and the processing 
flow. The WAS sends a query to the Query Redirector (QR) (1), then QR checks 
whether the given query is read or write. If the query is a write, it sends the query 
to the Consistency Maintainer (CM) (A). CM forwards the query to the origin 
database server (B) and performs the process to maintain cache consistency. 
If the query is a read, then it forwards the query to the Fragment Processor 
(FP) (2). FP references the Cache Dictionary (CD) to decide whether the result 
for the incoming query can be constructed based on cached fragments (3). A 
fragment is a query result stored in a cache. If fragments for the incoming query 
are found, FP retrieves the fragments (4). Otherwise, FP sends the query to the 
database server (a) and receives the result for the query (b). Then, it forwards 
both the query and the query result to the Cache Controller (CC) (c). CC inserts 
the query into CD and the query result into the Cache Pool (CP) (d). (When 
the cache does not have enough space to store new query results, CC executes a 
cache replacement algorithm.) FP constructs and sends the query result to QR 
(5). Finally, QR sends the query result to the WAS (6). 

CD is a collection of meta information about fragments stored in the Cache 
Pool (see Figure 3). Each entry stores meta information on a fragment (e.g., 
selection region, fragment size, and pointer to data). Entries are classified into 
different query groups according to the structure of queries. Then, each group 
is indexed by selection regions of the queries. The index is used to reduce the 
search time for fragments matching a query. 

The caching inherently incurs the inconsistency between cached results and 
an origin database. If a data element in an origin database is updated, the frag- 
ments derived from the updated data will be stale. WDBAccel includes CM 
which ensures the consistency. The primary goal of CM is to minimize the con- 
sistency overhead. It matches an update against the common parts of many 



1 The TPC-W benchmark is an industrial standard benchmark to evaluate the per- 
formance of database-driven Web sites. It models an e-commerce site (specifically, 
an online bookstore). 
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query processing flow 

additional processing flow when a cache miss occurs 
processing flow for update queries 



Fig. 2. WDBAccel architecture and processing flows 



queries, not individual queries. Thus, it avoids repeated matching of each query. 
In addition, CM ensures strong cache consistency by invalidating affected query 
results before the completion, i.e., the transaction commit, of a database up- 
date. For some pages, strong consistency is critical to service them always in an 
up-to-date version ( e.g . the cost of products in Web shopping sites). [5] describes 
our consistency mechanism in detail. 



3 Technical Details 

3.1 Derived Matching 

WDBAccel employs the derived matching to maximize the main-memory hit 
ratio, i.e., the rate of reusing query results cached in main memory. When WD- 
BAccel fails to find an exactly matching fragment, it tries to find one or more 
fragments from which a query result can be derived. We give an example of the 
derived matching. In this example, a query Q can be derived from the union of 
fragments F\ and F 2 although Q does not exactly match either F\ or F%. 
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Meta information on fragments 




Fig. 3. Cache Dictionary 



Example 1. Given fragments and a query as follows, 



F\ : SELECT * FROM ITEM 
WHERE I_PUB_DATE >= 


‘01/01/2003’ 


AND 


I_PUB_DATE 


<= '01/20/2003 


F 2 : SELECT * FROM ITEM 
WHERE I_PUB_DATE >= 


‘01/10/2003’ 


AND 


I_PUB_DATE 


<= '01/30/2003 


Q : SELECT * FROM ITEM 
WHERE I_PUB_DATE >= 


‘01/05/2003’ 


AND 


I_PUB_DATE 


<= '01/25/2003 



Q can be derived from the union of F\ and F 2 since the selection region of the 
union contains the selection region of Q. 

WDBAccel utilizes the selection region dependency in order to maximize 
finding the derived matching. A selection region dependency is a computational 
dependency in selection regions. In example 1, a query Q has the selection re- 
gion dependency on a union of a fragment F\ and a fragment F 2 in that the 
selection region of the union contains the selection region of Q. By investigating 
the dependencies among selection regions, WDBAccel is highly likely to find a 
derived matching. This is due to the characteristic of the queries used in many 
Web-based applications: selection regions from different queries with the same 
template tend to overlap each other and may form a hot range. In example 1, 
selection regions on I_PUB_DATE will frequently fall together near to the present 
time. This is because customers in an online bookstore prefer to select new books. 

When a query is matched by a derived matching, the process of the query 
result deriving follows the matching process. For the query result deriving, two 
operations, union and trim, are applied to matching fragments. The union opera- 
tion is used to merge the matching fragments and to eliminate tuple duplication. 
In the Example 1, by the union, the tuples of the matching fragments F\ and 
F 2 are merged and the duplication of the tuples (‘01/10/2003’ < I_PUB_DATE < 
‘01/20/2003’) is removed. We used the selection attributes of tuples to identify 
duplicate tuples. They are located in overlapping selection regions among the 
matching fragments. 

The trim operation is used to cut off the tuples that do not constitute a 
query result when matching fragments contain a query. In the Example 1, the 
tuples (‘01/01/2003’ < I_PUB_DATE < ‘01/05/2003’) of the matching fragment 
F\ and the tuples (‘01/25/2003’ < I_PUB_DATE < ‘01/30/2003’) of the matching 
fragment F 2 is cut off by the trim operation. In order to determine whether a 
tuple is a part of a query result, the attribute values of the tuple is compared to 
the selection predicates of the query. 
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3.2 Cache Storage 

To increase the utilization of a limited cache storage, it is crucial to avoid re- 
dundant storage of identical data. Query results can include identical tuples. In 
example 1, the tuples located in the overlapping selection region (l_PUB_DATE, 
‘01/10/2003’, ‘01/20/2003’) can be included in both fragments F\ and F 2 . As 
mentioned in section 3.1, overlaps among query results as above are common in 
many Web applications. 

To remove the redundant storage, WDBAccel identifies overlaps in selection 
regions of fragments and eliminates redundant storage of tuples. The storage 
policy of WDBAccel stores query results in the unit of tuples. Before storing 
each tuple in a new query result, it determines if the tuple already exists in the 
cache. This is done by comparing the values of the selection attributes of each 
tuple with those of the cached tuples. To speed up the comparison, the policy 
scans only the tuples of the fragments overlapping the new query result. Note 
that these fragments have already been retrieved in the query matching process. 
We refer to the Cache Pool adopting this policy as the Cache Pool in the unit 
of Tuples (CPT). 



3.3 Cache Replacement 



Hit ratio is improved if the cache stores the query results which are frequently 
accessed and consume less storage space. Thus, we evaluate the profit of a query 
result as follows: 



profit(f) 



popularity(f) 

S-COSt(f) 



where s-cost(f) is the storage cost of a fragment / and popularity(f) represents 
the popularity of a fragment /. When a cache space is full, the Cache Controller 
evicts the query result with the lowest profit value (called a victim). Usually, 
the last access time or the number of accesses are used for popularity . Under 
CPT, we consider that some tuples are shared among multiple fragments. We 
divide the storage cost of a shared tuple among sharing fragments. In this case, 
s-cost is computed as follows. 



s-cost(f) = size(ti)/n-frag(ti) 

ueT{f\ 

where T(/) is a set of tuples belonging to a fragment /, sizefti ) is the size of a 
tuple ti, and n-fragftf) is the number of the fragments containing a tuple ti. 



4 Experiments 

In this section, we compare the processing capability of WDBAccel with that of 
DBMS-based systems by measuring their throughputs. 
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(a) WDBAccel 



(b) DBMS-based cache system 



Fig. 4. Experimental setup 



Experimental Setup. Figure 4 (a) shows the setup for evaluating the WDBAc- 
cel system. The Query Generator emulates a group of Web application servers, 
generating queries. It runs on a machine with a Pentium III 1GHz, 512MB RAM. 
We implemented the prototype of WDBAccel which included all components and 
core functions described above. WDBAccel is deployed between the Query Gen- 
erator and the database server. WDBAccel runs on the machine with a Pentium 
III 1GHz, 512M RAM. For the origin database server, we used Oracle 8i with the 
default buffer pool size of 16MB. The database server runs on a machine with 
a Pentium IV 1.5GHz, 512M RAM. We populated the TPC-W database in the 
database server at two scales: 10K and 100K (cardinality of the ITEM table) 2 . 
All three machines run Linux and are connected through a 100Mbps Ethernet. 

Figure 4 (b) shows the experimental setup for evaluating the throughput 
of the DBMS-based cache system. The DBMS-based system conceptually con- 
sists of the cache-related components and the underlying DBMS. We used the 
simplified system by omitting the cache-related components. For the underlying 
DBMS, we used Oracle 8i. We also assume that the DBMS-based system achieves 
100% cache hit ratio. Thus, the Oracle database is fully populated with the en- 
tire TPC-W database. Note that the throughput measured under this simplified 
setup will be higher than that taken under a real situation using the DBMS- 
based cache systems. Both systems run on the Linux machine with a Pentium 
III 1GHz, 512MB RAM and are connected through a 100Mbps Ethernet. 

Workload Traces. We used for experiments the trace of the search-by-title 
query specified in TPC-W. This query searches for all the books whose titles 
include the keyword specified by users. It is the most frequently-used query in 
TPC-W; in an average case, its usage frequency constitutes 20% of the entire 
TPC-W workload. We believe that the query is frequently used in many e- 
commerce sites in general. The following is the query template of the query. We 
refer to the search-by-title query as the keyword query. 

SELECT TOP 50 I_TITLE, I_ID, A_FNAME, A_LNAME FROM ITEM, AUTHOR 
WHERE I_A_ID = A_ID AND I_TITLE LIKE ‘ °/„@Title°/ 0 ’ ORDER BY I_TITLE 



Performance Comparison Between WDBAccel and the DBMS-Based 
Cache Systems. The experiment was performed through two steps: the fill 
phase and the test phase. In the fill phase, we fill the cache space with fragments 

In TPC-W, the scale of database is determined by the cardinality of ITEM table. 
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by issuing 100,000 queries. This phase warms up the cache space so that the 
throughput can be measured under normal condition, i.e., within a normal range 
of cache hit ratio, during the test phase. The cache size of WDBAccel is set to 
range from 1% to 10% of the sum of ITEM and AUTHOR table sizes. ITEM and 
AUTHOR are the tables which are accessed by the keyword and the range queries. 
The sums of two table sizes corresponding to the 10K and 100K scales of TPC-W 
are around 10MB and 100MB, respectively. The buffer pool size of the DBMS- 
based system is set to 16MB and 128MB. 

Figure 5 shows the results for the keyword query. It shows that WDBAccel 
outperforms the DBMS-based system by an order of magnitude. This implies 
that reusing query results can significantly save the query processing cost. In 
ordinary database servers, the keyword query needs a linear search on the ITEM 
table. This is because the keyword query uses the LIKE operator, which includes 
wild card characters and does a pattern matching. Such a LIKE operator cannot 
benefit from the index structure on the attribute I_TITLE. On the other hand, 
in WDBAccel, the query processing time increases slightly with the number of 
fragments no matter what a query type is. Also, WDBAccel does not require 
any disk accesses. 

Another interesting observation is that the throughput of WDBAccel rapidly 
improves as the cache size increases. With the cache size up to 5%, the origin 
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database server is the bottleneck. In that case, the cache miss ratio (and thus 
hit ratio) is an important factor of the throughput. For example, if the cache 
miss ratio decreases from 2% to 1%, the throughput will be doubled. 

5 Related Work 

Recently, several query result caching systems have been reported in the context 
of dynamic Web content services. Form-based cache [10] is the first effort for 
a query result caching. It extended the URL-based proxy caching for active 
proxy query caching with limited query processing capability. The proposed 
framework could effectively work for a top-n conjunctive queries generated from 
the HTML forms. However, it only addressed keyword queries. Weave [13] caches 
data in the unit of XML and HTML pages as well as query results. Weave 
focuses on the declarative specification for Web sites through a logical model 
and a customizable cache system that employs a mix of different cache policies. 
DBProxy [1] employs a conventional DBMS for storing and retrieving query 
results like the DBMS-based caching systems. Therefore, its performance could 
be limited by the complex query processing of the underlying DBMS. The main 
focus of WDBAccel is to efficiently scale the performance of database-driven 
Web sites. It provides a framework for a high-performance query result caching 
and an efficient storage structure for it. 

The HTML caching has also been used to improve the performance of e- 
commerce sites. It selects cacheable data in the unit of the whole HTML page or 
HTML components which are parts of a HTML page. It then caches the data in 
front of Web servers or inside WAS’s. We call the former HTML page caching [8, 
3] and the latter HTML component caching [4, 7, 2]. The main advantage of the 
HTML caching is that the performance gain on a cache hit is better than that of 
others. This is because it saves much or all of the cost of HTML page generation 
as well as database processing. However, the HTML page caching is not effective 
in caching dynamic pages. HTML component caching can be effective to some 
extent in caching dynamic contents. A problem in the component caching is 
that it incurs high administration cost; cache administrators should go through 
a complex process of marking cacheable and non-cacheable units. 

6 Conclusions 

We have presented the design and implementation of a high-performance data- 
base server accelerator. WDBAccel improves throughput of the database process- 
ing, which is a major bottleneck in serving dynamically generated Web pages. To 
improve the performance, it reuses previous query results for subsequent queries 
and utilizes main memory as a primary cache storage. WDBAccel performs the 
derived matching to effectively find a set of query results required to construct a 
result of an incoming query. It employs the storage policy that reduces storage 
redundancy. In addition, the cache replacement policy takes into account storage 
costs of query results and overlaps among them. The experimental results show 
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that WDBAccel outperforms DBMS-based cache systems by up to an order of 
magnitude. 
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Abstract. Whenever product data is supplied by multiple product providers in a 
distributed electronic market and customer queries are submitted to all provid- 
ers of this electronic market, the query workload is one of the major bottle- 
necks. We present a technique which reduces query workload and communica- 
tion costs in distributed electronic market places involving multiple providers. 
The key idea is to guide customer queries through a network of brokers which 
aggregate the result information of previous queries, such that a query is posted 
only to those providers which may eventually provide products matching the 
query. A performance evaluation of our approach includes product data from a 
large number of car dealers as providers. It shows that our approach outper- 
forms the standard query broadcasting technique by a factor of up to 50 - de- 
pending on the set of customer queries considered. Therefore, we consider our 
approach to be an important improvement towards the reduction of the query 
workload within distributed electronic markets. 



1 Introduction 

1.1 Problem Origin and Motivation 

Whenever an electronic market integrates the distributed product data of several sup- 
pliers, and there exists a high frequency of queries submitted to this electronic market, 
then the query processing workload is one of the primary bottlenecks for the scalabil- 
ity of such a distributed market place. An important aspect of the query workload in 
the electronic market - and the focus of our contribution - is how many suppliers 
have to compute answers for each customer query. The standard approach, which we 
call broadcasting, is to send every query to every supplier. Whenever the number of 
suppliers and the number of customer queries is high, broadcasting may become in- 
feasible within the market place. In comparison, it may be considerably advantageous 
to submit queries for products only to those suppliers which can contribute to the 
answer of the query. Our key idea is to use previous queries and statistical informa- 
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tion of their results in order to submit new queries to only those providers which can 
contribute to the answer of a query. 

We have implemented our approach, and taken a market place for used cars as the 
data source for benchmark tests. Within our electronic market place for used cars, our 
providers are 60 car dealers which offer used cars over the market place. Our market 
is structured in such a way, that most of the car dealers have typical sets of offers, e.g. 
there are very few luxury car dealers, few dealers for vintage cars, and quite a lot of 
dealers offering cars produced by only a few companies. While the advantage of an 
electronic market is that a customer asking for a specific type of car can obtain rele- 
vant offers from all car dealers connected to the market place, the market places 
should avoid unnecessary query workload. Therefore, the market place should not 
send each customer query to each dealer, but only to those dealers which might pro- 
vide an offer matching the customer’s query. For example, a dealer offering luxury 
cars may prefer not to receive queries for the cheapest family cars every day, or a 
dealer offering vintage cars will not be interested in queries for a one year old BMW. 
Furthermore, most used car dealers located in Hamburg (northern Germany) will not 
be interested in a query from customers located in Munich (southern Germany) - 
since used car dealerships in general are local businesses - perhaps with the exception 
of luxury and vintage cars. 



1.2 Basic Assumptions and Problem Definition 

Within our market place, we assume that providers and customers agree on some kind 
of global relational schema of ordered domains, i.e, we assume that properties of 
products are represented as (attribute,value) pairs, where the attribute values are re- 
stricted by finite ordered domains. For example, an attribute like the construction year 
of a car is associated with an ordered range of values, e.g. the interval [1900:2004]. 
Whenever A1,...,AN are the N attributes of the global relational schema, which are 
used in common by providers and customers of the electronic market, and Dl, ..., DN 
are the finite discrete domains of Al, ..., AN (i.e. each attribute value of AI must be 
an element of Dl), we call Dl x ... x DN the schema of the electronic market, and 
each subset of D 1 x ... x DN is called a search space of the electronic market. 

For example, a query can restrict the interval for the construction year of a car to 
[2000:2004], and thereby look for product offers within a smaller search space. 

Within our approach to optimized query delivery, we are especially interested in 
search spaces which correspond to so called conjunctive point or range queries, i.e. 
queries which can be defined as follows. 

A point query is a query of the form 
Attribute = value. 

A range query is a query of the form 
Attribute < value 
or a query of the form 

Attribute > value. 

A conjunctive point or range query is a query of the form 

Condi tionl and ... and Condi tionN (withN>0) 

where each ConditionI (1<=I<=N) is either a point query or a range query. Note 
however that our approach still allows the customers to submit other (more general) 
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queries. For some, but not for all of these other queries our optimization applies as 
well. 

A search space is said to correspond to a conjunctive point or range query, if and 
only if the query condition Condi tionl and ... and Condi tionN is equiva- 
lent to true for every element of the search space, and it is equivalent to false 
for every element of the schema which is not an element of the search space. 

Whatever the internal presentation of the data at the provider’s side is, we assume 
that each provider transforms a query of the market place into its internal query for- 
mat, and conversely converts its data into the data format (i.e. attributes and values) 
supported and required by the market place. This can be achieved by using mapping 
technologies for product classifications (as defined in [4]), by general query mapping 
across heterogeneous data sources ([8]), or by XML based mappings (as defined in 
[16]). Similarly, we assume that each customer application transforms its queries in 
such a way that they use the common attributes Al, ..., AN of the electronic market, 
and conversely transforms the answers retrieved, i.e. from the data format of the elec- 
tronic market, into the data format required by the customer. 

A major requirement of our market place is that query processing supports the effi- 
cient evaluation of frequently asked conjunctive point or range queries. The state of 
the art approach to query processing is to broadcast customer queries to all providers. 
The broadcast approach does not use small search spaces in order to restrict a query’s 
application. Instead, queries are applied to the whole schema, and every query is 
therefore submitted to all the connected providers. In comparison, the goal of our 
approach is to send a query only to those providers which contribute to the answers of 
the query. By using our approach to optimized query delivery, query workload is 
significantly reduced compared to the broadcast approach. The possible degree of 
optimization depends on the given query, the set of supported search spaces and the 
distribution of product offers. The problem to be solved is how to reduce the overall 
query workload of the providers which are connected to a distributed market place. 
The query workload does not only include the amount of queries that are transferred 
from the customer to a provider, but also includes the queries which are needed to 
maintain the optimized query routing system. 



1.3 Related Work 

Our work is related to contributions in different fields including electronic markets, 
schema mapping, data warehousing and the use of aggregated data in query optimiza- 
tion. Within the field of electronic markets, a lot of work has been done in the area of 
structuring electronic market places, e.g. by product classifications (like eClass [11], 
ECCMA[13], ETIM [14], Edibatec [12]). These classifications are fixed, i.e. they are 
not dynamically adapted to queries. In contrast, our approach focuses on customer 
queries and performs dynamic clustering on product offers according to the most 
frequent customer queries. 

Within related contributions to schema mapping, two approaches to data and query 
translation can be distinguished. While the majority of contributions (e.g. [10], [1], 
[22]) map the data to a unique representation, we follow [7] and [8] to deliver the 
queries to those domains (or providers) where the data resides. Our approach is com- 
patible with this work and with further work on schema and query mapping for feder- 
ated databases [6] and data warehouses [25], [20], and even with mapping tools like 
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BizTalk Mapper [21], However in comparison to these contributions, we focus on the 
reduction of the query workload on the providers connected to the market place. 

In common with data warehouses, we use aggregate information on attributes in 
order to answer queries. While the majority of data warehouse approaches query for 
the aggregated information itself, our main use of aggregated information is to guide 
query processing, i.e. to deliver customer queries to the appropriate providers. 

Within our search structure, we maintain aggregated data that stores certain statis- 
tical information about the data distribution. Aggregated data has been used for query 
optimization by many other contributions, which can be classified into two categories 
[23]. While some contributions consider only the data distribution and focus on an 
estimation of query result size (e.g. [18], [19], [24]), we follow [9], [15], and [5] 
which also regard the query patterns. Furthermore, there are different methods used to 
represent data distribution. While parametric representations use mathematical formu- 
las (e.g., [9]), we use enclosing hyper-rectangles to represent tuple sets, like most 
non-parametric representations. Various solutions exist for how these rectangles are 
represented, divided and updated based upon changes in underlying data. R-Trees 
[17] and their further developments like R*-Trees [3], use B-Tree-based structures 
and heuristic algorithms for dividing rectangles, which results in a set of partially 
overlapping shapes. In terms of rectangle division, our approach is similar to contri- 
butions using multi-dimensional histograms (e.g. [5]). More precisely, it corresponds 
to recursive, non-overlapping histogram partitioning over multiple attributes. Like [2], 
[5] and others, but unlike typical R-Trees, we use incoming queries to adapt the col- 
lection of statistical information to a query profile. Unlike these solutions - which are 
aimed at selectivity estimation - and which are similar to R-Trees, we maintain exact 
statistics at all times, allowing us to guarantee complete, deterministic results despite 
routing queries only to a selected number of brokers or dealers. Therefore, unlike [2] 
or [5], we do not use the results of queries to directly acquire data distribution statis- 
tics, as this leads to a fragmentation of data space that is expensive to keep up-to-date 
upon changes to the underlying data. Instead, we only use the queries themselves to 
identify leaves in our decision tree that need further refinement. And we generate 
custom queries with specially chosen ranges to acquire statistical information for each 
newly generated node in our decision tree. Solutions based on sampling (e.g. [24]) 
also, in essence, generate custom queries to acquire statistical information, but they do 
not account for query profiles and, more importantly, they are not aimed at ensuring 
compatibility between different decision trees (as described in sections 3.3. and 3.4). 

In comparison to all of the other approaches, our data structure and the method for 
its modification combines an adaptation of brokers to different query sets (which is 
useful for selective routing) with a development of decision trees in such a way that 
allows bottom-up propagation of data updates (which is a pre-condition for the seal- 
ability of our solution to a large number of brokers). 

The remainder of the paper is organized as follows. Section 2 presents the key 
ideas of our approach, i.e., how a hierarchy of brokers can pass a query directly to the 
providers which can contribute to the answer of the query. Section 3 outlines when 
and how such a hierarchy of brokers is developed and maintained. Section 4 outlines 
the results of our performance evaluation, and Section 5 outlines the summary and 
conclusions. 
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2 Key Ideas 

2.1 Aggregated Information for Clusters of Product Offers 

An initial idea is to aggregate data about clusters of offered products within the elec- 
tronic market. A cluster of product offers is a set of product offers which match a 
certain conjunctive point or range query (or as we say, which are found in a certain 
search space). Note that a cluster is only a name for certain set of product offers, i.e. 
it contains no physical data referring to the product offers. Aggregation includes the 
counting of product offers found in a cluster, and the computation of minimum and 
maximum attribute values (e.g. for attributes like price, year, etc.) of product offers 
found in the cluster. This aggregated data can be used in order to estimate the number 
of positive answers to a query, and to support various types of queries (e.g. a query 
for the minimum price of a two year old BMW). 

One major goal of the use of aggregated information is to reduce the further sub- 
mission of unsuccessful queries to the connected providers. For example, if the mini- 
mum price for a set of stored offers is larger than the upper price limit given in a 
query, we are then sure that there can not be an answer which meets the query, i.e. 
based on the aggregated values alone, we can avoid to send the query any further. 

2.2 Hierarchical Clustering and Query Passing by Brokers 

A second idea is to use a hierarchical structure of so called brokers which control the 
flow and distribution of queries. Each broker is associated with its own search space, 
and via its search space, it is associated to its own cluster of product offers which are 
contained in the search space. For the purpose of query optimization, each broker 
stores the aggregated values for both the number of offers contained in its associated 
cluster, and the minimum and maximum attribute values found in each attribute of the 
product offers of its associated cluster. The brokers are arranged in a broker hierarchy 
in such a way that whenever two brokers exist in a parent-child relationship, the 
search space associated to the child broker is a subset of the search space associated 
with the parent broker, and thus the cluster of each child broker contains a subset of 
the product offers that are contained in the parent broker. At the bottom layer of our 
broker hierarchy, each provider is connected to the electronic market via a private 
bottom layer broker, which transforms all queries into the provider’s native data for- 
mat and transforms the answers back to the data format of the market place, i.e, each 
bottom layer broker operates as a proxy for the provider. 

Our goal is to identify the bottom layer brokers which can answer a customer’s 
query completely, but submit queries to as few associated providers (and brokers 
respectively) as possible. In order to minimize the submission of queries to bottom 
layer brokers, we introduce intermediate brokers which summarize the information of 
brokers that provide ‘similar’ offers to the given queries. The idea behind the intro- 
duction of intermediate brokers is to cluster similar offers of different providers de- 
pending on the queries. Clustering shall be performed in such a way, that those offers 
which match the same query are likely to be in the same cluster, and other offers are 
not in the same cluster. Each cluster of offers is managed by its own broker, such that 
the hierarchy of brokers supports hierarchical clustering of product offers. Having 
introduced hierarchical clustering, a query is always passed along the cluster hierar- 
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chy, i.e. from a parent broker to a subset of its child brokers. Within the following 
sections, we describe how queries are passed through the hierarchy of brokers. 

In comparison to query passing, the aggregated information, which is used in order 
to optimize query passing from a parent broker to a child broker, is always passed in 
the opposite direction, i.e. from the child broker to the parent broker. 



2.3 The Idea Behind Expanding the Search Space 

A third basic idea is to determine typical queries and to use queries in order to expand 
the search space as follows. The whole search space is divided into multiple search 
spaces in such a way that many queries can eventually be answered completely by 
submitting them only to a subset of these search spaces. 

For example, in the used car domain there may be one search space for old cars, 
i.e. cars built earlier than in the year 2000, whereas another search space contains new 
cars, i.e. cars built in 2000 or later. While queries for vintage cars can be answered by 
submitting them to the providers connected to the first search space only, queries for 
one year old cars can be submitted to only those providers which are connected to the 
second search space. 

Expanding the search space is an iterative process, and each expansion operation 
selects one search space and then divides this search space into two new search 
spaces. At the beginning there is only a single search space, i.e. every query belongs 
to this search space. However, over time the application of the expansion operation 
provides us with a decision tree for search spaces, which guides the passing of queries 
down the broker hierarchy to all the connected providers. 



2.4 A Decision Tree-Based Data Structure for the Distribution of Queries 

The data structure which hands over queries within a given hierarchy from one broker 
to lower level brokers is a decision tree (for an example see Figure 1 below). Each 
node of such a decision tree represents one broker. For each node (except the leaf 
nodes), an attribute-value pair is used in order to divide the search space managed by 
the current broker into two sub search spaces, managed by the two subsequent bro- 
kers. The idea behind the attribute-value pair (A,V) is to use the attribute value V in 
order to divide the range of attribute values given for the attribute A into two parts. 
When the expansion operation is carried out, offers are clustered into one of the two 
new search spaces, depending on their value V2 for the attribute A. If the value V2 
for the attribute A is less than or equal to V, the offer belongs to the search space 
represented by the left successor node of the expanded decision tree node (we say that 
the offer is clustered there). Otherwise, the offer has an attribute value which is larger 
than V, thus the offer belongs to the search space represented by the right successor 
node of the expanded decision tree node (and the offer is clustered there). 

Each node (A,V) of the decision tree is used to pass queries as follows. Whenever 
a conjunctive query contains a point query condition “A=V2” or a range query condi- 
tion “A<=V2” for a constant value V2 which is less than or equal to V, then the query 
is not passed to the right successor node of the actual decision tree node. Similarly, if 
the query contains a point query condition “A=V2” or a range query condition 
“A>=V2” for a constant value V2 which is greater than V, then the query is not 
passed to the left successor node of the actual decision tree node. Otherwise, if no 
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further optimizations apply, the query is passed to both successor nodes of the deci- 
sion tree, i.e. it is handled by both successor brokers. Whenever the decision tree node 
is a leaf node, the query is passed to all bottom layer brokers that are connected to this 
leaf node. 




Fig. 1. A (small) decision tree for the distribution of queries 

For example, Figure 1 shows a decision tree which contains an initial decision 
node (price, 10000) and a second decision node (year,2000) which only applies for the 
low cost cars. The first decision node is used to pass queries as follows. Whenever a 
conjunctive query contains a point query condition “price= Value” or a range query 
condition “price<= Value” for a constant Value which is less than or equal to 10000, 
then the query is not passed to the right successor node of the actual decision tree 
node. Similarly, if the query contains a point query condition ”price= Value” or a 
range query condition “price>= Value” for a constant Value which is greater than 
10000, then the query is not passed to the left successor node of the actual decision 
tree node. Otherwise, if none of the optimizations which are described in the next 
section apply, the query is passed to both successor nodes of the decision tree, i.e. it is 
handled by both successor brokers. 

Note that it is possible to keep multiple nodes (or even all nodes) of a decision tree 
within the same location, such that query passing via multiple brokers can be done at 
the same location. Therefore, besides the selection of the appropriate brokers, there is 
no overhead for query passing from one broker to a successor broker. 

2.5 Using Aggregated Information 

in Order to Reduce the Distribution of Queries 

As mentioned within Section 2.1, the brokers use aggregate values for counting the 
offers and for the storage of minimum and maximum attribute values of the offers of a 
cluster. 

Whenever a query is submitted to a broker that has counted the number of 0 prod- 
uct offers within its cluster (i.e. within the search space which is represented by the 
broker), query processing is stopped here and returns an empty set of answers. 

Furthermore, each broker uses its aggregated minimum and maximum attribute 
values of all its product offers as follows. Whenever a point query condition 
attribute=value 

searches for a value which is less than the aggregated minimum value or greater than 
the aggregated maximum value of the cluster, then query processing is stopped here, 
and the empty set is returned. A similar optimization is used for the upper and lower 
bounds for range query conditions. 
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2.6 Customers’ Local Brokers 

and Multiple Entry Points to Clustered Information 

Since the user queries may differ from each other, the appropriate clustering tech- 
nique may vary from query to query. In order to meet this requirement, we allow our 
system to support multiple entry points, i.e. multiple top-level brokers. These top- 
level brokers can be distinguished by the sets of attributes which they support. For 
example, one entry point may support queries within special price ranges whereas 
another entry point may support queries for a specific car model and year. Depending 
on the actual query, it may be preferable to use one or the other entry point to the 
system. Each entry point belongs to a different top-level broker, i.e, starts a new bro- 
ker hierarchy, which may be connected to a different set of intermediate brokers. In 
other words, there are multiple clustering hierarchies active in parallel within the 
whole system. Nevertheless, it is possible that the same intermediate broker is a sub- 
broker of different parent brokers within different broker hierarchies. 

Initially a customer query is handed over to the customer’s local broker, i.e. a bro- 
ker that resides at the site of the customer. The purpose of this local broker is to ana- 
lyse the query and to transform it into a conjunction 
QC and QR 

of two queries, where QC is a (possibly empty) conjunction of point or range queries 
and QR is a (possibly empty) remainder which can not be further transformed into a 
conjunction of a remainder and a conjunctive point or range query. Then, the attrib- 
utes found in QC are used in order to select an appropriate hierarchy of brokers in 
order to answer the query. Whenever there are multiple possible hierarchies which 
can be used, a hierarchy selection procedure selects the hierarchy which is most ap- 
propriate to the given query. 

Although our market place supports the efficient evaluation of conjunctive point or 
range queries, whenever they are currently supported by one of the hierarchies of 
brokers, our market place is not limited to this class of queries. Note that it is always 
possible to submit queries Q, the complexity of which goes beyond these conjunctive 
point or range queries. Whenever it is possible, Q is transformed into an equivalent 
form QC and QR , where QC is a conjunctive range query and QR is any remainder. 
In such a case, QC can be used in order to pass the query through the broker hierarchy 
to a sufficient subset of providers that answer the query completely. Note however 
that a query Q can also be answered, if it is not possible to transform Q into such an 
equivalent conjunction QC and QR. In this case, the query Q must be broadcasted 
to all suppliers, i.e. the query workload in our electronic market for the query Q is not 
worse than in the broadcast approach to query distribution. 



3 Hierarchical Cluster Construction and Dynamic Extension 
of Cluster Hierarchies 

3.1 When to Divide Which Attributes Within Which Search Space? 

Expansion operations are only performed on leaf nodes of the decision tree, i.e. on 
brokers which are direct predecessors of bottom layer brokers. We use the queries 
passed through such a leaf node in order to decide on which attribute is used in an 
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expansion step to divide the search space. Each expansion decision selects one attrib- 
ute to be the expansion attribute. An attribute can only be selected as an expansion 
attribute of a search space, if the search space allows for at least two different attrib- 
ute values for this attribute. 

The decision as to which attribute is chosen as an attribute for an expansion opera- 
tion depends on the search space, the number of queries submitted to that search space 
and the number of product offers found for that search space. This means that within 
different search spaces, the attribute selected for the expansion operation and the time 
at which the expansion operation is applied will usually be different. 

Each attribute which occurs within a query to a certain search space is a candidate 
for the expansion operation. However, the more frequently an attribute occurs within 
a query to this current search space, the more likely it will be used for an expansion 
operation. 

In order to decide which attribute is to be used for an expansion operation, we 
count the frequency by which an attribute is used within a query (called attribute 
usage ) for each attribute within each search space. Whenever the attribute usage in- 
creases beyond a given threshold and the number of answers within this data cluster is 
more than 0, we then divide this search space according to the given attribute. The 
two new leaf node brokers copy the attribute usage values for all attributes from the 
expanded node 1 . 

3.2 How Do We Divide the Search Space? 

Within the expansion operation, the range of the expansion attribute is divided into 
two parts which are disjointed but together cover the whole range of the attribute. 
While in general it is possible to divide an attribute range at any value, we decided to 
always divide an interval in the middle because this simplifies a possible later recom- 
bination of expanded search spaces 2 . Each bottom layer broker which has been previ- 
ously attached to the cluster, must now be assigned to one of the resulting clusters. 
Furthermore, after an expansion operation has been applied to a search space, we 
compute aggregated data for the two new sub-clusters which result from the expan- 
sion operation. More specifically, for both sub-clusters, we compute minimum and 
maximum values of all the attributes and count the number of answers found for the 
two new clusters. This computation can be done by an upwards propagation of the 
values found in the provider’s proxy clusters that are sub-clusters of these two clus- 
ters. 

3.3 Managing Multiple Hierarchies 

Whenever there are a lot of queries which are not yet supported by the given hierar- 
chy (or hierarchies) of brokers, it may be advantageous to build up one or more new 
hierarchies of brokers which support these types of queries. In order to explain what 



1 In order to avoid that the same attribute is used for expansion operations again and again, we 
adjust the attribute usage of the expansion attribute to a lower value (50%) after copying it 
from the expanded node. 

2 Note that we defined the domains of the attributes to contain discrete values. Thus, we are 
able to divide given intervals of concrete values. 
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happens, we define supported attributes and supported queries as follows. Whenever 
there is a decision tree which has an attribute Ai as the expansion attribute of its root 
node, then the attribute Ai is said to be a supported attribute. Let QC be the conjunc- 
tive point or range query part of a given query Q, and let Al, ..., AN be the attributes 
occurring in QC. If one of the attributes of QC is supported by a broker hierarchy, we 
call the queries Q and QC supported, because we can pass the query Q to this hierar- 
chy. However, if none of the attributes of QC is supported, we call Q unsupported. 

For each unsupported attribute, we count how many unsupported queries it occurs 
within. Whenever this number increases over a predefined threshold, the system in- 
stalls a new top-level broker which supports this attribute. In other words, this new 
top-level broker performs an expansion operation on this attribute and thereby gener- 
ates two sub-brokers. Thereafter, the appropriate bottom layer brokers are searched 
for and connected to both sub-brokers. 



3.4 Update, Insertion and Deletion of Product Offers 

Whenever a provider changes a product offer, the bottom layer brokers of this pro- 
vider have to check whether or not this influences the stored aggregated data for 
minimum and maximum values, and whether or not the changed attribute values of 
the product offer cross the border of the associated search space of the bottom layer 
broker. 

Whenever minimum or maximum values of a bottom layer broker are changed, the 
changes must be propagated to the higher level brokers. On the higher level brokers it 
is again checked as to whether or not the propagated changes modify the minimum 
and maximum values, and if so the changes are propagated further up the broker hier- 
archy. 

Whenever the change of attribute values of a product offer crosses the border of the 
search space of a bottom layer broker, then this update of an offer can be treated as an 
insertion followed by a deletion. 

Whenever a new product is offered by a provider which does not belong to a cur- 
rent search space associated with any bottom layer broker of this provider, then a new 
bottom layer broker for this new product offer has to be established at the interface of 
the electronic market to this provider. This is basically done be defining a new broker 
with all minimum and maximum attributes set to the current values found for the new 
product. Furthermore, this new broker has to be linked into each broker hierarchy, i.e. 
it has to be connected to that leaf node of each decision tree which represents a search 
space containing the product description described by this broker. In order to find the 
correct decision tree leaf node within each broker hierarchy, a point query is submit- 
ted to each broker hierarchy, and the data for the new product is used in order to spec- 
ify the attribute-value pairs of the point query. The same query passing mechanism 
which is also used for conventional queries is used in order to find the correct leaf 
node. The only difference is that these queries are not counted in order to find expan- 
sion attributes, and that the re-computation of sum and aggregate values is done on 
the fly. 

Whenever a product offer is deleted from a provider’s data source, it has to be 
checked for every bottom level broker, whether or not there are other offers within the 
same search space. If so, only the minimum and maximum values and the number of 
offers have to be adjusted and propagated in a similar manner as they are propagated 
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when an expansion operation occurs. If a search space becomes empty via a deletion 
operation, i.e, there is no product offer found in this search space any more, the num- 
ber of product offers of this search space is set to 0. Thereby, this search space be- 
comes a candidate search space which may eventually be combined with an adjacent 
search space. Adjacent search spaces are search spaces which have identical parent 
search spaces. Combining two adjacent search spaces can be regarded as the opposite 
operation of expanding a search space. The only purpose of combining two search 
spaces is to reduce the number of search spaces which have to be handled by the sys- 
tem. When memory is not a bottleneck, the combination of search spaces is not neces- 
sary. 



4 Performance Evaluation and Results 

4.1 Scenario and Data Used for Performance Evaluation 

The test scenario that we use to evaluate the proposed solution is the car retail busi- 
ness, with a real-world data set compromising approx. 15500 offers originating from 
60 dealers. The data set was retrieved from an existing, centralized internet market- 
place, A well-defined filtering was applied to the original data, eliminating small 
sellers (with less than 10 cars) and also achieving a close-to-normal distribution of the 
represented dealer inventory sizes. Test runs consist of a number of synthetically 
generated queries being sent to the system, while workload data is captured. Each test 
is based on a specific type of query, which is defined by the set of attributes in the 
query conjunction, and the distribution of values over the ranges specified for these 
attributes. The query distribution in most tests is equivalent to the data distribution - 
e.g., the most frequently offered brand would be the one most often asked for. In 
addition, queries with random attribute value distributions are tested separately (Test 
la in Table 1 at the end of Section 4) in order to see whether this would have any 
significantly different effect on results. The length of the tests has been chosen so that 
broker saturation is reached: a point at which there are virtually no further modifica- 
tions to the search space and thus the results remain constant - i.e., the standard de- 
viation of the workload over the last time interval is low. 



4.2 Results 

As our optimization goal is to reduce the query workload on the provider’s data - 
especially for a high workload of user queries, we vary the number of user queries 
received at the system entry point of the electronic market. For each dealer, we meas- 
ure how many queries reach this dealer during an interval of 10 user queries. The sum 
of these values constitutes the total query workload during the respective interval. 

Figure 2 depicts the total query workload over the course of tests 1-3, which are 
based on a distribution of attribute values equivalent to that of the data set. Up to 
about the first 500 customer queries, the system load will for short moments exceed 
the query workload of broadcasting (which we take as the 100% workload compari- 
son value). However, while at saturation, the query workload falls below 25% of the 
query workload of broadcasting. The final load for Test 3 is particularly low, i.e. ap- 
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proximately 2% of the load of broadcasting, which can be explained by the high se- 
lectivity of the underlying query type. 




Fig. 2. Total query workload for tests 1-3 

In order to better understand the benefits and costs of the system, the total load is 
further divided into net load, i.e. user queries actually routed to dealers, and overhead, 
i.e. queries submitted by the broker while expanding search spaces. To put the load 
factors into perspective, they are compared to the load of query broadcasting. Figure 3 
shows the breakdown of the total query workload for Test 1. The overhead graph (and 
by implication, the total workload) shows characteristic spikes that correspond to 
search space adjustments and associated broker-initiated queries - however, the spikes 
diminish with increasing saturation of the system. At the same time, the net load, 
which starts at 100% when no routing information is available, decreases continu- 
ously. 
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Fig. 3. Query workload breakdown for Test 1 



Optimized Query Delivery in Distributed Electronic Markets 63 



In addition, we capture the total memory usage of the broker as a second load vari- 
able. It is measured as a percentage of the total data set size (see Figure 4). At satura- 
tion, it is in the range of 7.5% to 10% for the various test cases. These values depend 
on the broker parameters used, which are equal for all test cases. 




Fig. 4. Memory usage depending on the number of incoming queries 

Note however that by tweaking these parameters, a different memory-consump- 
tion/query-workload ratio can be achieved. For example, for test case 1, the search 
space adjustments could be stopped at 5% memory usage (after 420 user queries), in 
which case the final system load would be 40% (see net workload in Figure 3), which 
is still a 60% saving compared to broadcasting. 

Table 1 shows a snapshot of total query workload and memory usage for all four 
types of queries that have been run, at a point when saturation was reached (1500 user 
queries processed). It includes the additional Test la, which is a variation of Test 1 
that uses equally distributed values for query ranges (e.g., each car make is requested 
equally as often). The small difference in the workload for Test 1 and Test la indi- 
cates that query value distribution has a limited effect on workload. 



Table 1. Summary of system performance at saturation (1500 user queries) 



Test # 


Query Attributes 


Workload 


Memory 

Usage 


1 


Make, Price 


9.7% 


8% 


2 


Make, Price, Engine Type 


10.0% 


9% 


3 


Make, Price, Engine Type, Year of Manufact. 


1.7% 


8% 


la 


Make, Price (equally distributed) 


11.3% 


7% 



5 Summary and Conclusions 

We have presented a query distribution system for an electronic market which is to 
the best of our knowledge the first system which combines the following ideas. Que- 
ries are filtered using hierarchically structured routing techniques via decision trees. 
This enables our system to reduce the amount of “unnecessary” queries, i.e. queries 
which are sent to a provider which can not contribute to the answer of the query. Fur- 
thermore, the decision trees which are used for query routing divide the domains for 
discrete attributes. They aggregate and store minimum and maximum values for at- 
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tributes of product offers, as well as the number of found product offers within each 
cluster, from the values given for its sub-clusters. Compared to the propagation of 
complex query results, aggregated data can be easily propagated within a hierarchy of 
brokers. The aggregated data is used in order to pre-select brokers and thereby clus- 
ters of product offers to which a given query has to be passed, i.e. this aggregated 
information helps to identify the providers which can contribute to the answer of the 
query. 

In order to evaluate the query workload saved by our approach, we have used over 
15000 product offers originating from 60 used-car dealers. The workload evaluation 
shows that our approach significantly outperforms the standard (broadcasting) ap- 
proach, when there is a high workload of queries submitted to the market place. Our 
approach performs better than the broadcasting approach, when there are more than 
1500 queries in total submitted to the market place. Depending on the kind of user 
queries, we can reduce the number of useless queries in such a scenario by 75% at a 
minimum and up to 98 % in the best case. Therefore, we consider our approach to be 
a significant improvement of query management in distributed electronic market 
places. 
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Abstract. Data aspects in workflow systems did not yet receive the 
same attention as process aspects. Various kinds of data are processed in 
workflow system: from case data to process data, from internal data to ac- 
cess to external databases or document exchanges in inter-organizational 
workflows. We propose a uniform treatment of all kinds of data in work- 
flow definition and provide abstraction mechanism which allow transpar- 
ent access to all kinds of data in a uniform way. We use XML as data 
access language in our workflow definition language WDL-X. The con- 
cept contributes to transparency of data location and logical and physical 
data independence of workflow systems. It facilitates the reuse of pre- 
defined activities and subworkflows on different data sets and eases the 
interaction of a workflow with its environment by abstracting from the 
actual representation of data. 



1 Introduction 

Data in Workflows. Workflow management systems (WfMSs) are not intended 
to provide general data management systems capabilities, although they have to 
be able to administer large amounts of data. According to the Workflow Man- 
agement Coalition (WfMC), which tries to come up with a generally accepted 
terminology, we should distinguish three kinds of data in workflows [20] . Work- 
flow control data are managed by a WfMS and describe workflow execution 
(e.g. control and data flow among activities), relevant internally for a WfMS 
and without a long-term impact beyond the scope of the current workflow (e.g. 
a termination state of an activity). Application data are managed by the ap- 
plications supporting the process instance and generally are never seen by the 
WfMS. Workflow relevant data are used by the WfMS to determine the state 
transitions of a workflow (e.g. transition conditions) and may be accessed both 
by the WfMS and the applications. 

There are also other proposals for classifying data in workflows. Case data 
contain data directly related to an individual case (e.g. insurance claim data in 
an insurance claim workflow instance), whereas master data contain data not 
directly related to an individual case, more general data, used in many different 
contexts (e.g. customer data). Structured data can be interpreted by a WfMS, 
while unstructured documents (e.g. pictures) cannot. 

These different data categories are not disjoint. For example case data can 
be workflow relevant data and master data might be application data. It is also 
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possible that some data are required by the WfMS (workflow relevant data) and 
applications (application data) in the same time. 

We generalize all these classes of data into business data and workflow con- 
trol data (or control data for short). Business data describe persistent business 
information necessary to run an enterprise. These data may be accessed by a 
WfMS, invoked applications and other systems independent from the WfMS. 
Control data has the same interpretation as in [20] . 

Data used within a workflow by a WfMS may be in different types and 
formats. Basically each product uses different solutions varying from the minimal 
set of built-in primitive types (number, string, date) [2] to user defined types (e.g 
HTML forms in Panta Rhei [7] or ER-diagrams in LEU [11]). 

Workflow Repositories and External Data Sources. Workflow reposito- 
ries are used practically by every WfMS to store their data. A process model 
repository stores information about process models (e.g. activity types, data 
types etc.) and a process repository stores all process specific information (con- 
trol data and part of the business data) [11]. As a repository could be used a file 
system managed by a WfMS (e.g. Staffware classic [18]), a RDBMS (e.g. Panta 
Rhei [7]) or an OODBMS [13]. The authors of the system Meteor [15] proposed 
the XML repository RepoX [17] for storing definitions of workflow processes in 
the form of XML documents. 

Control data are usually stored in a repository. Business data may be either 
controlled by a WfMS and stored in the repository or may be managed in ex- 
ternal systems (e.g. corporate database) and the WfMS maintains references to 
these data in the external systems. Access to these external data is frequently 
provided by automatic activities or other means which are not a part of the 
WfMS (e.g. Automatic Steps and Scripts in Staffware [18], system invoked ac- 
tivities in Panta Rhei [7]). In this case most of the activity programming will 
be related to updating external databases. An important drawback of this ap- 
proach is that these WfMS-external data can only be used indirectly as workflow 
relevant data, e.g. be queried for control decisions, because the WfMS needs a 
direct data access to determine the state transitions. 

We propose to provide a WfMS with a uniform and transparent access 
method to all business data stored in any data source. The WfMS should be 
able to use data coming from external and independent systems to determine a 
state transition or to pass it between activities as parameters. 

XML and Workflows. The importance of XML technology is increasing 
tremendously for workflow management. The WfMC presented two standards 
based on XML. The Wf-XML [21] supports interoperability between multiple 
workflow enactment services. XPDL [22] is designed to exchange workflow pro- 
cess definitions between different WfMSs. It allows the use within workflow def- 
inition data typed with XML Schema. However, it does not state how complex 
XML documents can be tested in expressions. Moreover, XPDL does not specify 
how one can access environmental data within a process definition. 

A fully XML based approach to workflow processing is presented in [14] . Par- 
ticipants of a workflow exchange messages described in XML language X-MSG. 
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The workflow itself is defined in another XML language X-IBP. The process 
control logic called XS-PCL is written in XSL. 

Another approach for processing XML documents in WfMSs is presented 
in [3]. The authors proposed to partition a single XML document into several 
meaningful segments. A segment is a unit of work that can be performed by an 
activity in a workflow process. 

XML is the main data format in many B2B standards (e.g. ebXML [5]). Also 
web services extensively use XML (e.g. BPEL4WS [1]). Methods for integrating 
WfMSs with standards for web services are becoming more important [16]. 

We propose to use XML as the main business data format at every stage of 
workflow processing. A WfMS should be able to test conditions on XML data 
to determine the state transitions, regardless of where these data are stored and 
maintained. Data-passing between activities should also rely on XML-standards, 
independent of whether these activities are internal to a workflow or exter- 
nal. Both goals aim at a seamless integration of intra-organizational and trans- 
organisational workflow and on location transparency of data. 

The remainder of this paper is organized as follows: In Section 2 we present 
the idea of uniform access to XML data in workflows with an example. Section 
3 contains a metamodel for describing XML data and data access plug-ins. In 
Section 4 we present an architecture for providing a WfMS with transparent 
access to XML documents stored in different data sources. Section 5 describes 
the mechanism of data access plug-ins. We draw some conclusions in Section 6. 

2 Uniform XML Based Data Access 

The work we present here is a continuation of our work in workflow systems. Our 
workflow system Panta Rhei [7] used a form-flow metaphor to provide access to 
workflow specific data. For interorganizational workflows [6] we saw the necessity 
to define adaptors to accommodate to different document definitions of partner 
organizations (clients, suppliers, etc.) and we then represented case data in XML 
[9] for interacting with other workflow or application systems. Now we propose 
XML as data representation and access methodology everywhere in the workflow 
definition where data is accessed. 

In a nutshell our approach is as follows: 

— We use XML schema to describe the structure of all data. 

— Activity and workflow definitions use XML-types for characterizing input 
and output parameter. These types are views on the actual data storage. 

— XPath is used for accessing data and evaluating conditions. 

— Access methods link the parameters to the actual data stores. 

— Access methods may be associated with parameters and variables at run- 
time. 

We illustrate the main concepts of our approach with an example. First we 
propose to use XML Schema types [10] to describe all kinds of data. XML Schema 
types describing business data are used in many workflows within an enterprise 
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Fig. 1. Example workflow definition as a workflow graph 



1 process processOrder(!N order : orderType, IN plugin : dataAccessPluqln ) 

2 documents invoice : invoiceType accessed By plugin, 

3 notification : notificationType; 

4 begin 

5 warehouseman collectOrder(order, notification); 

6 salesman producelnvoice(order, invoice); 

7 rf (order./order/deliveryMethod = "Priority Expres") 

8 warehouseman sendByDHL(order); 

9 else 

10 warehouseman sendByPost(order); 

11 end if : 

12 system notifyCustomer(notification); 

13 end ; 

Fig. 2. Example workflow definition in WDL-X 



(e.g. the same XML Schema type describing an invoice in all workflow defini- 
tions). The enterprise accesses several data sources (within or outside its orga- 
nization) of business data required in many workflow definitions. 

A workflow graph with an example workflow describing the shipment of an 
order is presented in Fig. 1. 

The workflow definition may be also represented in a script language called 
WDL-X. It extends the Workflow Definition Language (WDL) [7] by replacing 
all data definition and data access with XML technology. A script corresponding 
to the example is presented in Fig. 2. 

A document describing the order is passed to the process as an input param- 
eter (line 1). The parameter comes from the outside of the workflow. E.g. it may 
be passed by a business partner activating the workflow as a web-service, or the 
workflow is invoked as a subworkflow in some more complex process. 

Apart from the order two other documents ( invoice and notification declared 
in lines 2 and 3) are passed between the activities. All documents are typed with 
XML Schema (e.g. orderType). The definition of the simplified XML Schema 
type and an example of the order are in Fig. 3. 

Processes and activities can accept parameters in one of the following modes: 
IN - as an input parameter (read only), OUT - as an output parameter (write 
only) and INOUT - as an in- and output parameter (read/write). In the exam- 
ple are several activities, which accept as parameters XML documents in the 
described modes, e.g.: notifyCustomer ( INOUT notification : notificationType) 
or producelnvoice (IN order : orderType, OUT invoice : invoiceType). 
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1 <xs:complexType name- 'orderType"> 

2 <xs:sequence> 

3 <xs:element name="dispatchAddress" type="addressType"/> 

4 <xs:element name="invoiceAddress" type="addressType" 

5 minOccurs="07> 

6 <xs:element name="e-mail" type="xs:string"/> 

7 <xs:element name="deliveryMethod"> 

8 <xs:simpleType name="deliveryType"> 

9 <xs:restriction base="xs:string"> 

10 <xs:enumeration value="Priority Express7> 

11 <xs:enumeration value="Air Mail7> 

12 </xs:restriction> 

13 </xs:simpleType> 

14 </xs:element> 

15 <xs:element name="items"> 

16 <xs:complexType> 

17 <xs:sequence> 

1 8 <xs:element name="item" type="itemType" 

19 maxOccurs="unbounded7> 

20 </xs:sequence> 

21 </xs:complexType> 

22 </xs:element> 

23 <xs:element name="subtotal" type="xs:float7> 

24 <xs:element name="postage" type="xs:float7> 

25 <xs:element name="totalBeforeVAT" type="xs:float7> 

26 <xs:element name="VAT" type="xs:float7> 

27 <xs:element name="total" type="xs:float7> 

28 </xs:sequence> 

29 <xs:attribute name="orderNo" type="xs:string7> 

30 </xs:complexType> 



1 <order orderNo="026-6462982"> 

2 <dispatchAddress> 

<name>John Doe</name> 

4 <street>Anystreet 7</street> 

5 <city>Anycity</city> 

6 <ZIP>A-007</ZIP> 

7 <country>Neverland</country> 

8 </dispatchAddress> 

9 <e-mail>john@any.org</e-mail> 

10 <deliveryMethod>Priority Express</deliveryMethod> 

10 <items> 

11 <item> 

12 <name>Krzysztof Penderecki - Orchestral Works</name> 

1 3 <price>4.99</price> 

14 <orderedQuantity>1 </orderedQuantity> 

1 5 </item> 

16 <item> 

1 7 <name>Henryk Sienkiewicz - Quo Vadis?</name> 

18 <price>24.50</price> 

19 <orderedQuantity>1 </orderedQuantity> 

20 </item> 

21 </items> 

22 <subtotal>29.49</subtotal> 

23 <postage>8. 1 9</postage> 

24 <tota I Before VAT >30.89</totalBeforeVAT > 

25 <VAT >6.79<A/AT > 

26 <total>37.68</total> 

27</order> 



3 



Fig. 3. XML Schema type definition and an example XML document instance 



First a human actor warehouseman executes the manual activity collec- 
tOrder. This activity produces a new notification instance. Afterwards a sales- 
man has to produce an invoice. Depending on the delivery method indicated in 
the order it is shipped by DHL or by post. At the end the system automatically 
notifies the customer per e-mail about the shipment. 

The example in Fig. 2 illustrates the new concepts of our approach. The 
WfMS works with valid XML documents typed with XML Schema. These data 
are accessed in a uniform way regardless of where they are stored. This is achieved 
by special, replaceable and reusable wrappers of external data sources called data 
access plug-ins (see sec. 5). A data access plug-in is passed to the example work- 
flow definition in line 1 as an input parameter plugin. This plug-in is later used 
to access an XML document of type invoiceType (line 2). Documents may also 
be stored locally in a process repository. In such a way the document notification 
is accessed (line 3). 

Data access plug-ins increase productivity and flexibility of the WfMS. We 
are able to specify a workflow definition and say where the data come from during 
the instantiation of this definition. Data access plug-ins may also be declared as 
constants in WDL-X, passed as elements of any document the process receives, 
or predefined during the instantiation in a process control sheet, which includes 
control data of the process. 

To control the flow of the workflow an XPath expression may be evaluated 
on XML documents as in the line 7 of the example. Documents tested in this 
way can be accessed by a data access plug-in. Thus workflow relevant data do 
not have to be stored in the process repository anymore. 
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3 Metamodel 

We propose a new metamodel (Fig. 4) for static workflow schema aspects. It 
describes a workflow and related actors, activities and data. It focusses on data 
aspects by describing XML documents, XML Schema types, data access plug-ins 
and forms. XML Schema types describe XML documents passed between activi- 
ties and accessed by various data access plug-ins. XSLT transformations provide 
transformations between various types. Simple data items of XML documents 
can be addressed with XPath expressions and be used to define conditions. 

Activities are either (external) workflows, elementary or complex activities. 
Elementary activities are either automated or manual activities. Complex activ- 
ities are composed of other activities. The type of a complex activity describes 
its control structure ( seq for a sequential, par for parallel and cond for condi- 
tional). Agents are responsible for executing activities. An agent may be a user, 
or a role. 

Activities can accept parameters in one of the access modes. The XML doc- 
uments are passed as parameters between activities. Each XML document has 
one root element of one XML Schema type. Types are either simple or complex. 
A complex type can have attributes and elements. A new type can be derived 
from an other type. Attributes are always of a simple type, elements can be 
either of a simple or a complex type. XML documents can be accessed by data 
access plug-ins. Each plug-in is capable of accessing XML documents of one or 
more XML Schema types. A document of one type can be transformed to a doc- 
ument of another type by a transformation between the source and the target 
XML Schema type [8,12]. 

A simple data item is an attribute or an element of a simple type. The 
simple data item can be addressed within an XML document with an XPath 
expression. The simple data item (or a combination of several items) is used to 
uniquely identify the XML document in a collection of many documents of the 
same type. Simple data items are also used in conditional complex activities 

Manual activities need some way of presenting XML data to a human actor. 
We propose to use forms defined in XForms [4]. XForms can be parameterized 
and accept XML documents as input and produce XML documents as output. 

4 Proposed Architecture 

We propose a new architecture of a WfMS which supports the usage of XML 
documents at every stage of workflow processing and allows the WfMS to trans- 
parently access many sources of business data via data access plug-ins. The 
architecture is presented in Fig. 5. It includes new modules responsible for stor- 
ing information about XML Schemas, managing data access plug-ins and for 
transforming between XML documents. 

The Data Schema Manager registers XML Schemas according to the meta- 
model presented in section 3. A workflow designer may use registered types in 
a workflow specification. This part of the WfMS manages also the information 
about relations between types and the transformations between different types. 




Transformation 
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Fig. 4. Workflow metamodel 
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Fig. 5. Proposed architecture 



The Data Transformation Manager uses information about the registered 
XML Schema types and the transformations provided by the Data Schema Man- 
ager and transforms XML documents when necessary in workflow execution (e.g. 
to accept XML documents from business partners). 

The Worklist Manager is responsible for worklists of the human actors and 
for the interaction with the client software (worklist handlers). The Program 
Interaction Manager calls programs implementing automated activities. It may 
also accept calls from the external systems e.g. Wf-XML messages. The Workflow 
Engine provides operational functions to support the execution of instances of 
business process, based on the process definitions [20]. The Plug-in Manager is 
described in section 5. 

5 Data Access Plug-Ins 

The Data Plug-in Manager controls data access plug-ins. A data access plug-in is 
a replaceable and reusable wrapper of an external and independent data source. 
Each plug-in must be registered with the manager together with it’s applicable 
XML Schema types. The requirements for the data access plug-in are: 

— Creation, selection, update, and deletion of an XML document in a collection 
of many documents of the same XML Schema type. 

— Evaluation of an XPath expression on a selected XML document. 

The data access plug-in can be wrapped around any data source if it provides 
the described functionality, e.g. files from a file system, XML views over relational 
or object-relational databases, or native XML databases. Thus the WfMS can 
access legacy databases easily. 












74 



Johann Eder and Marek Lehmann 



Consider the following frequent scenario: an enterprise has a large database 
with the customer data which are used in many processes. In our approach the 
company defines a XML Schema type describing customer data and implements 
a plug-in, which wraps this database and retrieves and stores customer data in 
XML format. The XML type has to be registered in the Data Schema Manager 
and the plug-in in the Data Plug-in Manager. From now on in all workflow 
definition the plug-in can be used for accessing customer data. This approach 
has many advantages: 

— Business data from external systems are directly accessible by the WfMS. 
Thus these data can be also used as workflow relevant data. 

— Both XML Schema type and the data access plug-in are reusable. They can 
be used in many workflow definitions. 

— This solution is easily evolvable and maintainable. If the customer data have 
to be moved to a different data source, it is sufficient to use another plug- 
in (see sec. 2). The workflow definition and activities remain basically un- 
changed. 

— All data in the workflow are in XML form. This simplifies data transforma- 
tions with many technologies developed for XML. Thus the seamless inte- 
gration with other partners and B2B standards is much easier. 

With improved technology for XML-views over relational data, which allows 
not only retrieving but also updating data [19], the implementation of plug-ins 
will become easier. Modern type system will also allow for the creation of generic 
plug-ins on the definition level which are then adaptive to the actual sources in 
the implementation. 

6 Conclusions 

The main contributions of the presented approach for uniform access to data in 
workflows are: 

— All data in workflows (application data, process data, external data sources, 
etc.) are described, represented and processed in a uniform way. 

— We offer a simple and transparent mechanism for accessing data stored in 
many different data sources (workflow repository, external systems). 

— Seamless integration with external systems can an be achieved by exchange 
of process and application data in XML format. 

— XML data types and data access plug-ins can be reused in many workflow 
definitions. 

— Reuse of subworkflows and activities is made easier and is no longer prohib- 
ited by differences in data representation. 

The concept and the architecture we propose strives for achieving true physi- 
cal and logical independence of process and data. The abstraction represented in 
exchangeable plug-ins for data access frees workflow definitions from the acciden- 
tiality of representation formats. Besides the obvious advantages for intra- and 
interorganizational exchange of data and documents, maintenance and evolution 
of workflow systems will benefit considerably. 




Uniform Access to Data in Workflows 



75 



References 

1. T. Andrews, F. Curbera, H. Dholakia et al.: Business Process Execution Language 
for Web Services (BPELfWS), Version 1.1. http://ifr.sap.com/bpel4ws/ 

2. M. Ader: Workflow and Business Process Management Comparative Study. Vol- 
ume 2. Workflow & Groupware Strategies, June 2003 

3. H. Bae, Y. Kim: A document-process associacion model for workflow management. 
Computers in Industry 47, pp. 139-154, Elsevier, 2002 

4. M. Dubinko, L. L. Klotz, R. Merrick, T. V. Raman: XForms 1.0. W3C Recom. 

5. ebXML Technical Architecture Project Team ebXML Technical Architecture Spec- 
ification vl.0.4. http://www.ebxml.org/ 

6. H. Groiss and J. Eder. Workflow systems for inter- organizational business pro- 
cesses. ACM SIGGROUP Bulletin, Dec. 1997. 

7. J. Eder, H. Groiss, W. Liebhart: The Workflow Management System Panta Rhei. 
I 11 : A. Dogac, L. Kalinichenko, T. Oszu, A. Slieth (Eds.): Workflow Management 
Systems and Interoperability, Springer- Verlag 1998 

8. J. Eder, M. Lehmann: Composition of Transformations for XML Schema Based 
Documents. Short paper at ADBIS 2003 

9. J. Eder, W. Strametz: Composition of XML-Transformations. EC- Web 2001 

10. D. C. Fallside: XML Schema Part 0: Primer. W3C Recommendation 

11. V. Gruhn, M. Schneider: Workflow Management based on Process Model Reposi- 
tories. IEEE Conference on Software Engineering 1998 

12. M. Lehmann: Exploiting Generalization for the Composition of Transformations of 
XML Schema Based Documents. CAiSE Forum 2003, in CEUR-WS.org/Vol-74/ 

13. C. Liu, X. Lin, X. Zhou, M. Orlowska: Building a Repository for Workflow Systems. 
31th Conference on Technology of Object-Oriented Languages and Systems, 1999 

14. M. Lee, S.-H. Kang: XML-based Automatic Workflow Control for Integrated Busi- 
ness Process. Submitted for publication. 

15. J. A. Miller, D. Palaniswami, A. Sheth, K. Koschut, H. Singh: Web Work: Meteor ’s 
Web Based Workflow Management System. JUS (10), Kluwer 1998 

16. M. Sayal, F. Casati, U. Dayal, M.-Ch. Shan: Integrating Workflow Management 
Systems with Business-to-Business Interaction Standards. ICDE 2002 

17. M. Song, J. A. Miller, I. B. Arpinar: RepoX: An XML Repository for Workflow 
Design and Specifications. Technical Report at University of Georgia, 2001 

18. Staffware pic: Staffware Technical Overview. Issue 1. October 2001 

19. L. Wang, M. Mulchandani, E.A. Rundensteiner: Updating XQuery Views Published 
over Relational Data: A Round-trip case study. XML Database Symposium, 2003 

20. Workflow Management Coal.: Terminology and Glossary. 1999, WFMC-TC-1001 

21. Workflow Mngt. Coal.: Interoperability Wf-XML Binding. 2001, WFMC-TC-1023 

22. Workflow Mngt. Coal.: Workflow Process Definition Interface - XML Process Def- 
inition Language (XPDL). 2002, WFMC-TC-1025 




Formal Verification 
of BPEL4WS Business Collaborations 



Jesus Arias Fisteus, Luis Sanchez Fernandez, and Carlos Delgado Kloos 

Telematic Engineering Department 
Carlos III University of Madrid 
Avda. Universidad, 30 
28911 Leganes, Madrid, Spain 
{ jaf , luis , cdk}@it ,uc3m. es 



Abstract. Web services are a very appropriate communication mech- 
anism to perform distributed business processes among several organ- 
isations. These processes should be reliable, because a failure in them 
can cause high economic losses. To increase their reliability at design 
time, we have developed VERBUS, a framework for the formal verifica- 
tion of business processes. VERBUS can automatically translate busi- 
ness process definitions to specifications verifiable in several available 
tools. It is based on a modular and extensible architecture: new process 
definition languages and verification tools can be added easily to the 
framework. The prototype of VERBUS presented in this work can verify 
BPEL4WS process specifications, by translating them to Promela. The 
Promela specifications are verified with the well known model checker 
Spin. In this paper we describe the general architecture of VERBUS and 
how BPEL4WS specifications are translated and verified. The explana- 
tion is completed by describing what types of properties can be verified 
and providing an overview of the implementation. 



1 Introduction 

Inter-organisational business processes are a key technology for business to busi- 
ness collaborations. Nowadays many enterprises have automated their internal 
business processes with workflow technologies. They have now a new challenge: 
the automation of their collaborations with partner enterprises, in open and very 
dynamic environments, to accelerate their business in a cost-effective manner. 
Web services are a promising technology to support these type of collabora- 
tions [1,2]. It is an XML-based middleware technology that provides RPC-like 
remote communication, using in most cases SOAP over HTTP. 

Web services provide a state-less communication mechanism: WSDL can 
specify remote operations and their input and output parameters, but not the 
relations between several operations. Business processes have a state. Therefore, 
new languages are necessary to execute business processes on top of Web ser- 
vices. Several languages have been used to model these business processes [2]. 
They are often called choreography languages, because they specify the order in 
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which the activities of the process must be executed. The most important are 
BPEL4WS [3], BPML [4] and ebXML BPSS [5], Among them, only BPEL4WS 
is specific for Web services. The Web Services Choreography Working Group of 
the World Wide Web Consortium (W3C) is also currently developing a new Web 
services choreography language. 

Complex business collaborations require the specification of complex business 
processes. Specificating complex processes is error prone, due to concurrency in 
the execution of activities, the possibility of communication errors, faults in 
remote systems, etc. Enterprises can only trust in this technology if the cor- 
rectness of the processes can be ensured, because a failure in them can cause 
high economic losses. In this work we present VERBUS ( VERification for BUSi- 
ness processes), a system for automatic verification of business processes using 
model-checking. Its main objective is to help process designers to ensure the 
correctness of the defined processes. The current prototype receives an input 
BPEL4WS process specification and a set of properties that the designer wants 
to verify. Then the system automatically translates the specification to a for- 
mal specification language and verifies it using a model-checker. If a property is 
found to be false, the system gives a counter-example to the designer. VERBUS 
is modular and extensible: new process definition languages and verification tools 
can be easily added to the framework. 

Several works have been done previously on business processes verification. 
Woflan [6] is a Petri-Net based verification tool. It can perform verifications on 
workflow definitions, and was integrated with several commercial workflow man- 
agement systems. In [7] formal semantics are defined for UML activity diagrams 
to allow the verification of workflow processes defined with these formalisms. 
It uses the SMV model checker. A framework for the verification of Web ser- 
vices is proposed in [8] . It can perform analysis on a Web service described with 
DAML-S by translating the description to a Petri-Net based model. 

None of these works can be applied to BPEL4WS processes. The results 
of these works are specific, both in terms of process modelling language and 
verification tool. However, VERBUS proposes a framework in which several pro- 
cess definitions languages and verification tools can be integrated, based on a 
common intermediate formal model. This formal model is very simple, but can 
represent complex semantics like the fault handling mechanism of BPEL4WS in 
a straightforward way, as showed in the next sections. 

This paper is organised as follows. Section 2 makes a brief introduction to 
BPEL4WS. Section 3 describes the main architecture of VERBUS. Section 4 
explains how VERBUS translates BPEL4WS processes to its formal model. Sec- 
tion 5 explains the possibilities that VERBUS offers for performing verifications. 
Finally, the main conclusions of this work are summarised. 



2 Modelling Business Processes with BPEL4WS 

Business Process Execution Language for Web Services (BPEL4WS) [3] is an 
XML notation for specifying business process behaviour based on Web services. 
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Layer 3: DESIGN MODEL 
Layer 2: FORMAL MODEL 
Layer 1: VERIFICATION MODEL 

Fig. 1 . Architecture of the VERBUS framework 



BPEL4WS 


other languages 


VERBUS formal model 


SPIN (promela) 


other verification tools 



It was developed by Microsoft, IBM, BEA, Siebel Systems and SAP. It allows the 
specification of executable processes and business protocols. An executable pro- 
cess defines the behaviour of a participant in a collaboration. A business protocol 
defines the message exchange of all the participants involved in a collaboration. 

BPEL4WS provides basic activity structuring (sequential and parallel com- 
position, conditional execution and loops), variables, hierarchical activity com- 
position with scopes, Web services based communication (invocation of remote 
Web services operations and providing Web services operations to remote sys- 
tems), event handling and fault and compensation handling mechanisms. 

Activities are executed in the context provided by a scope. Each scope con- 
tains activities and data. All the activities contained in a scope share the same 
context. Scopes can be hierarchically nested. The root of the hierarchy is the 
process, that can be viewed as a special scope. It can contain any number of 
children scopes. Each scope can contain also children scopes, and so on. Scopes 
store data in variables, that can be accessed by any activity contained by this 
scope, or any scope nested in it. 

Scopes are also important for fault and compensation handling. The fault 
handling mechanism is very similar to the Java exception handling mechanism. 
An activity can throw faults to notify an error in its execution. Faults can be 
handled by a scope or, if not, they are re-thrown to the next enclosing scope. 
Section 4.4 explains this in detail. The compensation mechanism allows the spec- 
ification of transactional behaviour for business processes. Each scope can define 
a compensation handler to make the rollback of the actions that were executed 
by the scope. This handler is executed when the scope has successfully completed 
but it must be compensated due to faults that occurred in other scopes. 

3 The VERBUS Framework 

VERBUS is a modular and extensible framework for the verification of business 
processes. It proposes an architecture with three layers, as showed in Fig. 1. 
The design model (layer 3) deals with the design of business processes, using 
specification languages like BPEL4WS or BPML, for example. The formal model 
(layer 2) deals with the specification of processes using a formal model. VERBUS 
defines its own formal model for this layer, based on Finite State Machines 
(FSMs). The verification model (layer 1) deals with the verification of business 
processes. Several general-purpose verification tools can be placed in this layer, 
such as Spin [9] or SMV [10]. 
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Layer 3 specifications are translated to formal (layer 2) specifications us- 
ing automatic translation tools. There is one tool for each layer 3 language. 
Layer 2 specifications are also translated to verification languages using auto- 
matic translation tools. There is one tool for each verification language, because 
each verification tool has normally a specific input language. Layer 2 is an in- 
termediate layer that increases the modularity and extensibility of the system, 
by disconnecting the design and verification layers. Thus, only one translation 
tool is needed when introducing a new verification tool in the framework, and 
it will be available to specifications defined in any language in the design layer. 
The same applies to the introduction of new layer 3 languages. 

The current prototype of VERBUS implements two translation tools. One 
of them translates a BPEL4WS process specification to a formal specification. 
The other translates a layer 2 specification to a Promela [9] specification, that 
can be verified with the model-checker Spin. 

3.1 The VERBUS Formal Model 

The formal model used in the layer 2 of VERBUS is based on FSMs. It is briefly 
presented here. Its formal definition is given in [11]. 

A process is composed by a set of attributes and a set of transitions. At a given 
moment, each attribute has a value within a set of possible values. The value 
of all the attributes of the process at a given moment establishes its state. The 
process progresses from one state to another by means of transitions. A transition 
is a pair of states (origin and destination) that defines a possible progress of the 
process. The process starts at an initial state. Then it fires transitions, until it 
reaches a state that is not in the origin of any transition. This state implies the 
completion of the process and is called a final state. 

The FSM of a business process has normally many transitions. In order to 
avoid defining them explicitly, VERBUS represents them with functional tran- 
sitions. A functional transition is defined by two predicates: domain and action. 
The domain defines a set of states that are origin of transitions. The action de- 
fines how these origin states change to obtain the final states. Therefore a func- 
tional transition represents a set of transitions that share a similar behaviour. 

The concepts of entity, entity type and activity are introduced as notational 
elements, to make specifications more readable. However, they do not affect the 
basic formalism. An entity type is a group of typed Helds (boolean, enumerated 
or integer types). An entity is an instance of an entity type. Each field of an 
entity type generates as many attributes as times its data type is instantiated. 
An activity is a logical unit for grouping related functional transitions. 



4 Translating BPEL4WS to the VERBUS Formal Model 

The translation of BPEL4WS specifications to the formal model is the most 
complex functionality of VERBUS. This section summarises how it is done. 
The translation of sequences and the fault-handling mechanism were selected 
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<xsd: complexType name=" Order "> 

<xsd : sequence> 

<xsd: element name= "product Id" 

type="xsd: string" /> 

<xsd: element name=" colour" > 

<xsd : simpleType> 

<xsd : restriction base="xsd : string "> 
<xsd: enumeration value="white"/> 
<xsd: enumeration value="red"/> 
<xsd: enumeration value="blue"/> 
<xsd: enumeration value="black"/> 

(...) 

</xsd: complexType> 

Cmessage name="OrderMessage"> 

<part name="urgent " type="xsd: boolean" /> 
<part name= " order " type= " tns : Order " /> 
</message> 

<variable name="order" 

messageType="tns : OrderMessage "/> 



enttype OrderMessage ■( 
urgent : boolean ; 

order productld: abstract; 

order colour: enum (white, red, blue, 

black) ; 

> 

entity order: OrderMessage; 



Fig. 2. Mapping between BPEL4WS variables and VERBUS entities 



as representative examples of how the translation is performed. The current 
prototype of VERBUS can translate also any of the other activities. In the web 
page of the VERBUS project (http : //www . it .uc3m. es/jaf /verbus) there are 
several examples that show how VERBUS translates these other activities. 

4.1 Variables 

BPEL4WS variables are mapped to VERBUS entities. Each variable is an in- 
stance of a data type defined by a WSDL message, an XML element or an XML 
Schema type. First, the data type is transformed to an entity type. Then it is 
instantiated as an entity. Simple data types are transformed to VERBUS data 
types if possible (boolean, enumerated and integer), or declared as abstract 
otherwise. Complex data types are transformed by recursively transforming their 
components. Fig. 2 shows an example. The message type OrderMessage has two 
parts: urgent and order. The part urgent is a simple type and so it is translated 
to a boolean field in the VERBUS entity type. The part order is a complex type: 
the sequence of the elements productld (string) and colour (enumerated data 
type). It is translated to two fields, one for each element. 

4.2 Activities 

The execution of each BPEL4WS activity instance is controlled by a life-cycle. 
Depending on the type of activity two different life-cycle types were identified 
in this work. The general life-cycle is used for activities that can have han- 
dlers (process, scope and invoke). The simple life-cycle is used for the other 
activities. Both life-cycle types are represented in Fig. 3. 

Each activity is mapped to an entity and several functional transitions. The 
entity represents the state of the activity in its life-cycle. The functional transi- 
tions represent the way the activity can progress through its life-cycle and how 
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Fig. 3. Life-cycle for activities. The general life-cycle is on the left and the simple life- 
cycle is on the right. States are labelled with uppercase letters and transitions with 
lowercase letters. States containing a black dot can be final states of the life-cycle 

it affects to the attributes of the process. The concrete functional transitions of 
each activity depend on its activity type. However, there are several rules that 
are common to almost all activities. 

Activity instances have always a begin transition, that represents the begin- 
ning of their execution. Its domain represents the preconditions of the activity. 
Normally it is a condition that checks the state of other activities (depending 
on the type of its parent activity), and its own state (it must be not_started). 
Its action changes the state of the activity to running. 

Activity instances have normally a complete transition also, that represents 
the end of their execution. Its domain checks that the activity is in running state. 
Depending on the activity type, it may include other conditions. Its action puts 
the activity in complete state, and can change also attributes of the process to 
represent the effects of the activity execution. 

If an activity has a non-deterministic behaviour then it is normally modelled 
with several mutually-exclusive functional transitions. Each of them represents 
a different behaviour of the activity. Examples of non-deterministic behaviour 
are pick activities, activities that can throw faults, receive or invoke activities 
that can receive messages with different data, etc. 

4.3 Sequence Activity 

The BPEL4WS sequence activity can contain one or more inner activities, that 
must be executed in sequential order. Given an activity in the sequence, it can 
begin its execution only if its preceding activity has been completed. The se- 
quence itself is completed when its last inner activity has been completed. 
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<sequence name="main"> 


activity main_act_2 { 






Creceive name="init" .../> 


transition begin { 






<switch name=" switch" > . . .</switch> 


domain: {main_act_2_lc . 


st at e=not .started} 




<scope name="end">. . .</scope> 


action: {main_act_2_lc . 


state=running}} 




</sequence> 


transition complete { 








domain: {(main_act_2_lc 


. state=running & 






end_act_ll_lc 


. state=completed) } 






action: {main_act_2_lc . 

> 

activity init_act_3 { 


state=completed}} 






transition begin { 








domain: -C(main_act_2_lc 


. state=running & 






init_act_3_lc 


. state=not_started) } 






action: {init_act_3_lc . 

transition complete {...} 

> 

activity switch_act_4 { 


state=running}} 






transition begin { 








domain: {(init_act_3_lc 


. state=completed & . . 


.)> 




action: -C(switch_act_4_lc 
... > 

activity end_act_ll { 


. state=running & . . 


.)» 




transition begin { 








domain: {(switch_act_4_lc 


. state=completed & 


...)> 




action: -(end_act_ll_lc . 

... > 


state=running}} 





Fig. 4. Mapping a BPEL4WS sequence. The examples is abbreviated to highlight the 
most important conditions and transitions 



This behaviour is modelled by adding a condition to the domain of the begin 
transition of each inner activity and a condition to the domain of the complete 
transition of the sequence activity. The condition added to the first inner activ- 
ity states that the sequence activity must be in running state. The condition 
added to the other inner activities states that the previous activity must be 
in completed state. The condition added to the complete transition of the se- 
quence activity states that the last inner activity must be in completed state. 
Fig. 4 shows an example. 

4.4 Fault Handling 

BPEL4WS has a powerful fault-handling mechanism. The process, scope and 
invoke activities can contain fault handlers. When a fault is thrown in a given 
activity, a handler in the immediately enclosing scope, process or invoke is 
selected, based on the fault name and variable type. Before the handler is exe- 
cuted, all the running inner activities of this scope are cancelled. If no handler 
is appropriate, then the whole scope is cancelled, and the fault is re-thrown 
to the next enclosing scope. A fault that reaches the process level causes the 
cancellation of the whole process. 

To manage the fault-handling mechanism, several attributes are added to 
the general life-cycle entity type. One of them is boolean and its value is true 
when a fault has occurred in the activity. The other is enumerated, and its value 
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specifies which is the selected handler when a fault has occurred. The activity 
contained in each fault handler checks these variables as a precondition for its 
execution. 

The cancellation mechanism is needed to implement fault-handling. It is 
implemented in VERBUS in this way: a scope that must be cancelled puts itself 
in fault-cancelling state. Its inner activity has a transition that cancels itself 
if the scope is in fault-cancelling state. In a similar way, if this activity has 
inner activities, they detect this cancellation and cancel themselves, and so on. 
The scope has a transition that puts itself in faulted state when none of its 
inner activities is running. At this moment the activity of the fault handler is 
allowed to start its execution. When this activity completes its execution, the 
scope puts itself in completed state. 

4.5 Prototype Implementation 

The current prototype of VERBUS is mainly composed by a BPEL4WS to 
VERBUS translator and a VERBUS to Promela translator. It is based on the 
BPEL4WS specification version 1.1 [3]. It was developed in Java and uses the 
open source libraries Xerces, Xalan and WSDL4J. The prototype works in com- 
mand line, but a graphical user interface is currently under development. It will 
incorporate a graphical editor of BPEL4WS processes. 

The main lack of the current prototype is the compensation mechanism, 
because of the complexity associated with it: a copy of all the variables must be 
saved for each completed activity instance, because they must be compensated 
using the value that variables had when they were completed. While loops even 
make this more complex, because multiple instances of each inner activity can be 
created. This feature will be handled in future versions of VERBUS, by storing a 
copy of attributes for each completed activity. This will increase the complexity 
of the verification, and therefore a configurable parameter will be added to limit 
the maximum number of activity instances. 

5 Verification of Processes 

The main goal of VERBUS is the verification of business process specifications. 
VERBUS allows the modeller to state properties that must be true for a given 
process specification, and checks whether these properties are true or false for 
it. If some property is found to be false, VERBUS gives a counterexample. From 
the point of view of the formal layer, properties are expressed with boolean 
predicates about the value of the attributes of the process. The current prototype 
of VERBUS can verify several types of safety and liveness properties: 

— Invariants : an invariant is a predicate that must be true in every reachable 
state. From the point of view of the BPEL4WS process, invariants look like 
for every state if the activity named “init” is running, the part “ urgent ” of 
the global variable “ order ” must have a false value. The counterpart property 
in the formal model layer is: ! (init_act_3 . state=running & order). 
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— Goals: a goal is a predicate that must be true in every reachable final state. 
I.e. the predicate must be true whenever the process stops its execution. 
VERBUS adds automatically one goal to ensure that the process and all 
the activities are in a valid final state of their life-cycle (not_started, 
completed, cancelled or compensated) when the process reaches a final 
state. Thus any dead-lock or process block is detected. Goals like when the 
process completes its execution the part “ urgent ” of the global variable “ or- 
der ” must have a false value can detect functional errors in specifications. 

— Transition pre and post-conditions: given a transition, a pre-condition 
(post-condition) is a predicate that must be true always immediately before 
(after) the execution of the transition. An example is: immediately before 
the activity named “init v completes its execution the part “ urgent ” of the 
global variable “ order ” must have a true value. 

— Activity reachability analysis: VERBUS can detect transitions that can not 
be executed in any trace of the process. Thus activities that can never be 
started are detected, for example. 

— Properties defined with LTL: VERBUS can check properties expressed in 
LTL ( Linear Temporal Logic). Using LTL the modeller can specify temporal 
causalities like if the part. “ urgent ” of the global variable “ order ” has a true 
value, then sometime in the future it must have a false value. 

Formal layer specifications can be translated to Promela in a very straightfor- 
ward way. The generated Promela specifications have a main do loop, in which 
all the transitions of the process are defined. The domain of each transition acts 
as a guard, and appears before the action. There is an else statement that 
breaks the loop when no transition can be selected (process completion). After 
the loop, there is an assertion for each goal property. Assertions for invariants are 
placed in a concurrent Promela process. Assertions for pre and post-conditions 
are placed before or after the action of each transition. In [11] this translation is 
explained in more detail. 

6 Conclusions 

This work presents VERBUS, a modular and extensible framework for automatic 
business process verification. It proposes an architecture with three layers: the 
design layer, the formal layer and the verification layer. The formal layer is a 
business process specification model based on the FSMs formalism. It disconnects 
process description languages and verification languages. Process definitions (de- 
sign layer) can be automatically translated to specifications in the formal layer. 
These specifications can be automatically translated to specifications in the ver- 
ification layer and verified using verification tools. 

Works had been done previously on business processes verification, but they 
can not be applied directly to BPEL4WS compositions. They use specific process 
description languages and verification tools. On the contrary, VERBUS provides 
an open framework in which several process description languages and verifica- 
tion tools can be integrated. 
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The implementation of a prototype of VERBUS has demonstrated the fea- 
sibility of the framework. The prototype is mainly composed by two translation 
tools. The first one translates BPEL4WS specifications to the formal model. The 
second one translates formal model specifications to Promela specifications, that 
can be verified using Spin. The VERBUS formal layer can model the flow control 
primitives commonly used in business processes. It is even expressive enough to 
model the complex fault handling and cancellation mechanisms of BPEL4WS. 

As future work, this first prototype will be completed by implementing the 
BPEL4WS compensation mechanism. Support for new process specification lan- 
guages like BPML and verification tools like SMV will be added to VERBUS. 
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Abstract. Web service technology extends the existing web infrastructure by 
transforming it from a repository of documents to a source of services. As the 
number of web services increases, the provision of the appropriate service pub- 
lication and discovery framework is of paramount importance for exploiting the 
full potential of the web service technology. This paper presents the principles, 
the functionality and the design of PYRAMID-S, a scalable framework for uni- 
fied publication and discovery of semantically enhanced services scattered 
around heterogeneous Registries. It uses a hybrid peer-to-peer topology to or- 
ganize Registries based on domains. In such a topology, each Registry retains 
its autonomy, meaning that it can use the publication and discovery mecha- 
nisms as well as the ontology of its choice. Furthermore, the discovery of ser- 
vices is based on QoS characteristics in order to enable service selection. 



1 Introduction 

Web service (WS) technology is a collection of emerging standards, tools and plat- 
forms that extend the existing web infrastructure to a business-oriented, transactional 
platform, where multiple web services may interoperate to provide information, trans- 
act business and generally take action for users or agents, dynamically and on de- 
mand. Web services may be published in various registries with incompatible publish- 
ing and discovery mechanisms (e.g. UDDI Registries [12], DAML-S Registries 
[7] [8], ebXML Registries [5]) resulting in cumbersome service publication and dis- 
covery. This situation is being aggravated by the increasing number of services. 
Therefore, the location and selection of suitable services becomes a critical issue and 
the provision of the appropriate service publication and discovery framework is of 
paramount importance for exploiting the full potential of the web service technology. 

At present, the most prevalent standard for WS publication and discovery is the 
UDDI [12] specification. However, the effectiveness of UDDI is limited due to a 
number of shortcomings that are related to the following issues: 

Service Description and Matchmaking: UDDI is mainly used in combination with 
WSDL [IT]. WSDL is an XML grammar for specifying the syntactic aspects of a web 
service such as what it does, where it is located and how it is invoked. The only se- 
mantic information about services is provided by using the various standard UDDI 
taxonomies (related industry, products or services offered and geographical region). 
However, this semantic information cannot be used for inferring relationships during 
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searching; this is due to the fact that the UDDI search services are limited to keyword 
search on certain fields such as name, identifier or taxonomy. Furthermore, the cur- 
rent UDDI model limits the discovery of a service to functional requirements and thus 
it cannot address questions related to non-functional requirements such as; “will the 
web service meet my performance requirements such as 2 ms response time?” 

Scalability: UDDI partially addresses the scalability issue through the use of multiple 
replicated nodes in the same UDDI registry. Apart from fault tolerance, the use of 
multiple nodes decreases the number of publication/discovery requests per node. 
However, this scheme does not reduce the number of entries per node, thus resulting 
in limited scalability and high cost for the operators of the UDDI nodes. UDDI Ver- 
sion 3 introduces the notion of multiple registries that may share data among them- 
selves with the knowledge that keys (unique identifiers of each entity within a regis- 
try) remain unique. In this way, entities in a private registry, for instance, can now be 
copied into another private registry for broader exposure or into a public registry, for 
public consumption. However, this entails the following disadvantages: (a) further 
increase of the number of service entries in each registry and (b) further communica- 
tion overhead for keeping consistent all copies of the shared entities. 

In this paper we propose a framework (PYRAMID-S hereinafter), which addresses 
the above limitations and in addition it supports seamless federation of heterogeneous 
registries. PYRAMID-S could be considered as an improvement to UDDI and as a 
contribution to the ongoing evolution of web service technology. The rest of the paper 
is organized as follows; Section 2 gives an overview of PYRAMID-S, while Section 3 
describes the PYRAMID-S functionality and design. Finally, we conclude with a 
discussion section. 

2 Overview of PYRAMID S 

PYRAMID-S is a framework that addresses the existence of heterogeneous service 
description and discovery mechanisms as well as the lack of semantics and the seal- 
ability issue of UDDI. The lack of semantics is tackled by using a number of ontolo- 
gies which are presented in the following section. The heterogeneity and scalability 
issue are addressed by the adoption of a layered architecture presented in section 2.2. 

2.1 PYRAMID-S Ontologies 

In PYRAMID-S we address the challenge of semantics by using four different types 
of ontologies: the Standard Domain Ontology (SDO), the Registry Domain Ontology 
(RDO), the Domain Classification Ontology (DCO) and the QoS Ontology. 

Standard Domain Ontology (SDO): This ontology reflects abstract concepts and rela- 
tionships in a particular application domain. It has two parts: the Operation part and 
the Data part. The Operation part models major action types and thus helps to deter- 
mine the type of operations that each web service performs. For example the Opera- 
tion part of an SDO for the Loans Services domain may include concepts such as 
CreditScoreCalculation and CreditProfileCheck. The Data part incorporates concepts, 
their properties, and relations among concepts in a particular application domain. For 
example an SDO for the Loans Services domain (gray part of Fig. 1) may include 
concepts such as Loan, Bank, LoanAmount, ServiceFee and InterestRate. The SDO of 
a specific domain is the default ontology of the PYRAMID-S framework for that 
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domain. The Registries conforming to that SDO use this ontology for the semantic 
publication and querying of the web services they hold. 

Registry Domain Ontology (RDO): Registries may either adopt the SDO or use their 
own domain ontology (RDO). In the second case the Registry operator has to provide 
a mapping from its own ontology to the SDO. Ontology Mapping is the alignment of 
entities (concepts, attributes, relations etc.) in one ontology with those of another 
ontology, so as to capture shared meaning (Fig. 1). The necessity of having RDOs and 
the mapping from RDO to SDO stems from the fact that no global enforcement on the 
use of ontologies is possible in highly autonomous environments. 




EquivalentClass 

EquivalentClass 



EquivalentClass 



Fig. 1. A Sample SDO and a sample mapping from SDO to RDO. 



Domain Classification Ontology (DCO): It maintains relationships among domains 
and mappings of each Registry of the PYRAMID-S to one or more domains. In addi- 
tion, it stores properties of Registries, such as the access URL, the Registry provider 
details, the access URL of the RDO and the mapping from RDO to SDO (in case of 
non-conformance to SDO), as well as the constraints in accessing that Registry. It also 
stores the relationships between domains and SDOs. Fig. 2 shows a sample DCO in 
the form of a tree. Information regarding the Registries in a DCO may be expressed as 
a set of tuples d | =<R | , D ; , A ; >, where R ; is the access URL of a Registry, Dj is a do- 
main, and Aj are the properties of the Registry. The tuples are identified by the com- 
bination of R | and D ; , since a Registry may be mapped to more than one domains. 
This means that for any x, y (T x iT y => R x ^R y v D x -£D y ). Information regarding the 
domains in the DCO may be expressed as a set of tuples V ; = <D ; , SDO ; >. 

QoS Ontology: Based on previous studies [9] [10] and our experience in the WS area, 
we have constructed a QoS ontology composed of the following dimensions: response 
time, cost, availability and security. This ontology is used in combination with SDO 
for the semantic publication and querying of web services. 



2.2 The PYRAMID-S Layered Architecture 

One of the main goals of our work is to provide a scalable framework for seamless 
federation of heterogeneous registries. Scalability can be attained by distributing ser- 
vices in domain specific Registries. This enables more pertinent service discovery as 
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the selection of a domain Registry works as a first filter in the discovery process. For 
example, if a Registry is related to the “Loan Services” and “Insurance Services” 
domains, it will maintain web services specific to those domains and queries for such 
type of services can be routed to it. The seamless federation can be achieved by add- 
ing a layer of unification over the heterogeneous registries. 




Fig. 2. A Sample DCO. 



On the basis of the above considerations we propose a framework, which consists 
of three layers depicted in the right part of Fig. 3. The bottom layer, i.e. the Registries 
Layer, already exists while the two other layers (in gray background), namely the 
Gateways and the Routers Layer, are introduced by us in order to achieve our goal. 

The Registries Layer consists of a number of Registries provided by diverse Regis- 
try operators. Each Registry retains its autonomy and the two other layers of 
PYRAMID-S act as a meta-Registry that controls and supports access to the Regis- 
tries. PYRAMID-S accommodates any kind of web service Registry such as DAML- 
S [7] [8] based Registries, UDDI Registries and ebXML Registries [5], 

The Gateways Layer consists of a number of servers that are known a priori to the 
clients. The servers of this layer function as entry points to the PYRAMID-S system 
and provide to the users a single unified view over heterogeneous Registries. 

The Routers Layer consists of a number of servers that provide routing service to 
the Gateways in order to forward the queries/advertisements entered in the Gateways 
to the appropriate domain Registries. 

PYRAMID-S is based on a peer-to-peer network, which provides the infrastructure 
for the distributed nodes of PYRAMID-S to communicate with each other. The peer- 
to-peer topology renders PYRAMID-S scalable as it allows Routers to easily adver- 
tise and unadvertise themselves to the Gateways. More importantly, this topology 
ensures that there is not a single point of failure in the Routers layer. 

Each node, depending on the layer it belongs, plays a particular role in the peer-to- 
peer network. Each peer of the Routers Layer plays a role similar to that of an index 
server in a semi-centralized peer-to-peer network where the peers communicate with 
the index server in order to obtain a reference to the data/processing that is available 
on the network. This implies a hybrid peer-to-peer network. In the following, we 
present in detail the functionalities provided by each layer of PYRAMID-S frame- 
work. 
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Fig. 3. The PYRAMID-S Architecture. 

3 Detailed Description of PYRAMID-S 

3.1 PYRAMID-S Functionality 

Gateways Layer. As mentioned before the peers of this layer function as entry points 
to the PYRAMID-S framework. User actions are directed to any peer of this layer, 
which then, depending on the user action, interacts with either a Router peer or both a 
Router peer and one or more Registry peers and returns a reply. User actions may 
vary from service queries or advertisements to modifications of the DCO, depending 
on the role of the user. PYRAMID-S distinguishes three different types of users: 

• Simple users, who publish and discover web services 

• Registry operators, who, in addition to publishing and discovering web services, 
can add/remove Registries from PYRAMID-S and update Registry information 

• Domain administrators, who can update the DCO and the SDOs, besides publish- 
ing and discovering web services 

Browsing and searching through the DCO and SDOs is available to all types of us- 
ers and greatly facilitates most of their actions presented in the following paragraphs. 

Service Publication. When a user wishes to register a service in PYRAMID-S, s/he 
searches for the desirable Registries using the appropriate interface provided by the 
Gateways. The Registries can be found using various criteria, such as the Registry 
Provider, the Domain or combination of them. Then s/he selects the desired Registries 
from a list of Registries conforming to her/his criteria. The Gateway provides the user 
an interface for entering the Service Advertisement (SA), the structure of which is 
described later. The SA is then converted to a form understandable by each Registry 
and is forwarded to it. Finally, the user is informed about the result of his/her request. 

When users publish a service they provide the WSDL document describing it (we 
assume that all types of Registries use the WSDL as a basis for service description, as 
it is the most widely accepted standard) and semantically annotate each operation and 
its I/O parameters with the corresponding concepts found in the SDO. These semantic 
annotations accompanied with the QoS metrics and the service provider information 
constitute the Semantic Service Description (SSD). Thus, the Service Advertisement 
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(SA) consists of two parts, the SSD and the WSDL document, where SSD = <SP, 
OPs, Is, Os, QoS> (SP: service provider information, OPs: set of service operations 
specified using concepts found in the Operation part of the SDO, Is/Os: set of in- 
put/output parameters specified using concepts found in the Data part of the SDO, 
QoS: quality of service metrics). 

Service Discovery. Service discovery is based on a Service Query (SQ) that specifies 
the requirements about a service to discover: SQ = <SN, SD, SP, OP, Is, Os, QoS>, 
(SN: service name, SD: service textual description, SP: service provider information, 
OP: requested operation specified using concepts found in the Operation part of the 
SDO, Is/Os: set of input/output parameters of the requested operation specified using 
concepts found in the Data part of the SDO, QoS: quality of service metrics). 

Service discovery may be performed either on a specific Registry or on all the Reg- 
istries of a specific domain. This is accomplished through an appropriate user inter- 
face, provided by the Gateway Layer, which enables the service requestor to specify 
the SQ describing his/her needs. The SQ is built by using concepts of the SDO that 
the Gateway peer retrieves from a Router. The service requestor may also specify 
relative weights corresponding to the QoS characteristics. The SQ is submitted to the 
selected Registries after it has been properly translated. Each Registry uses its own 
matchmaking algorithms. The results of the Registries are returned to the Gateway 
which consolidates and ranks the results based on the user’s predefined weights. 

Registry Management. Registry operators may use the respective interface provided 
by a Gateway in order to insert/update or delete a Registry and its associated proper- 
ties in the PYRAMID-S system. The DCO is presented to the Registry operator in 
order to associate his/her Registry to the appropriate domain. The user input is trans- 
lated into one of the following operations on the DCO: 

• Insert(T x ): Registry R x is stated to be related to domain D x and to provide its ser- 
vices with A x properties (T x =<R x , D x , A x >). This operation is valid only if there is 
no T y eDCO: R y =R x a D y =D x . After the completion of the operation, DCO is 
DCO+{T x }. 

• Delete(T x ): Registry R x is no longer related to domain D x . This operation is valid 
only if T x e DCO. After the completion of the operation DCO is DCO-{T x }. 

• Update(T x ,A x '): The properties of Registry R x for domain D x are updated to A x \ 
This operation is valid only if T x eDCO. After the completion of the operation 
DCO is DCO-{T x }+{(R x ,D x , A x ')}. 

If a mapping from RDO to SDO is needed, the registry operator provides it by us- 
ing the appropriate Gateway interface. In this interface, the RDO and SDO are re- 
trieved from a Router and represented as taxonomy of concepts in a tree structure. 

Domain Administration. Domain administrators may update the DCO with the addi- 
tion of new domains or renaming of existing ones. Also, new SDOs may be created 
and existing ones may be modified with the addition of new concepts. Domain dele- 
tion in the DCO and concept renaming or deletion in the SDOs are not allowed, as 
they would introduce inconsistency regarding registered Registries and services. 

Routers Layer. This layer consists of a number of peers, each holding a copy of the 
DCO, SDOs, RDO to SDO mappings, as well as a copy of the QoS ontology. 
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Through the use of the DCO, the Routers provide routing service to the Gateways in 
order to forward the queries/advertisements entered in the Gateways to the appropri- 
ate domain Registries. The peers of this layer ensure high availability, protection and 
consistency of the routing information, which is information regarding Registries and 
their mapping to domains. If any Router is disconnected from the network, the routing 
service is not affected unless this peer is the last Router of the peer network. 

Registries Layer. This layer consists of a number of Registries that may be heteroge- 
neous due to different choices at the physical level (different DBMSs), logical level 
(different data models) and conceptual level (different ontologies). The peers of this 
layer are responsible for getting the service advertisement (SA) or the service request 
(SQ) from the Gateway and for performing the necessary actions. 

3.2 PYRAMID-S Design 

In this subsection we present the design of PYRAMID-S Gateways and Routers and 
describe how the aforementioned functionality is offered. We do not present the de- 
sign of Registries since they are autonomous entities participating in PYRAMID-S. 

Gateway Design. There are two ways of accessing PYRAMID-S: through a Web 
Server GUI or through a web service (WS interface). Each Gateway peer provides 
both interfaces (Fig. 4). By using the Web Server GUI end-users may perform service 
queries/publications or administration tasks. The Web Server GUI interacts with the 
WS interface of a Gateway peer, through which the overall functionality of 
PYRAMID-S becomes available. 

Actions requested by end users are translated into the appropriate SOAP messages, 
which are sent to the WS interface of a Gateway. The Web Server GUI is enhanced 
with additional facilities, such as visual navigation of the DCO and SDOs and auto- 
matic generation of GUIs for web service invocation. Furthermore, the Gateway WS 
interface is publicly available and may be accessed by other client applications. 

A Gateway peer is a composite web service utilizing four constituent web services, 
namely the Ontology Accessor, the Mediator, the Publisher and the Finder. Fig. 4 
depicts the interdependencies among these services, as well as external communica- 
tions. 




Fig. 4. Detailed design view of PYRAMID-S. 
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The Ontology Accessor communicates with Router peers in order to create, retrieve 
or update the DCO and SDOs. 

The Mediator is a utility service that transforms service queries/advertisements 
from PYRAMID-S representations to Registry specific representations. Service dis- 
covery results, deriving from Registries, are also transformed reversely. Implementa- 
tion of the transformation operations depends on syntactic and semantic conventions 
used by the Registry, as well as the support for QoS characteristics. Therefore, a dis- 
tinct Mediator service is needed for every type of Registry incorporated in 
PYRAMID-S, which may result in having more than one Mediators in each Gateway. 

The Publisher performs a service publication given the Service Advertisement, the 
domain(s) where the service is to be published and optionally a set of selected Regis- 
try peers for the publication. If no Registries are specified, the most appropriate ones 
are located by the Publisher through the DCO (provided by the Ontology Accessor) 
and used. The appropriate Mediators are then used for translating the service adver- 
tisement to Registry interpretable forms. 

In a similar manner, the Finder accepts Service Queries, the domain(s) to search 
for the service and optionally a set of selected Registry peers for the discovery. If this 
optional parameter is omitted, the Finder uses the DCO (retrieved by the Ontology 
Accessor) to locate the Registries that belong to the specified domain(s). Discovery 
requests are sent to the selected Registries, after the necessary transformations per- 
formed by Mediators. Discovery results are reversely transformed by the Mediators 
and forwarded to the Finder which ranks them according to the weighted value of 
their QoS characteristics, according to user preferences. 

Router Design. Each Router has a web service interface that allows building, retriev- 
ing and updating the DCO and SDOs. Insertions, deletions and updates of the DCO 
(actions hereinafter) may be performed by any Router (upon request by a Gateway) 
and are propagated to all other Routers. Therefore consistency of the DCO should be 
assured during conflicting operations, performed by the same or distinct Routers 
within a short time-frame (i.e. the time needed for an operation to be propagated to 
the whole peer group). There are two cases of having conflicting operations: 

• when two or more Routers attempt to rename the same domain of the DCO 

• when insertions, deletions and updates are performed on the same couple of Rx 

and Dx, i.e, on the same Tx 

Our conflict resolution mechanism uses a request/reply/notify scheme incorporated by 
the Routers, as described in [14]. This may be seen as a distributed lock management 
scheme, where each requesting Router requests permission, by the Routers peer 
group, to perform an operation on a specific part of the DCO. In case of a conflict, the 
resolution mechanism allows at most one Router to proceed with its requested opera- 
tion. 



4 Discussion 

In this paper we have presented PYRAMID-S, a framework that enables seamless 
federation of heterogeneous Registries. More specifically, PYRAMID-S entails the 
following advantages: 
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• It is an open platform as it integrates various types of WS Registries achieving 
thereby better interoperation without affecting their specifications and autonomy. 

• The introduction of Gateways as an entry point to PYRAMID-S abstracts the inter- 
face of heterogeneous Registries into a single unified view. Thus, users are allevi- 
ated from the burden of handling the diversion between various technologies since 
they can uniformly publish or discover web services. 

• The use of syntactic, semantic as well as QoS information about a service enables 
higher recall, improved precision and better result ranking. 

• The accommodation of a great number of Registries, Routers and Gateways solves 
the scalability problem. Registry categorization according to DCO helps in narrow- 
ing the search context and improves performance. Besides, in PYRAMID-S the 
exposure of private Registries to public consumption is much simpler in compari- 
son to the copying mechanism proposed in UDDI version 3. Overall system per- 
formance may be tuned by adapting the number of Routers and according to busi- 
ness requirements. 

• The service-oriented design and implementation of PYRAMID-S allows the use of 
its modules for building other applications. 

The most relevant work to ours is the METEOR-S system [6], It uses an ontology- 
based approach to organize Registries, enabling domain based semantic classification 
of WS. The main differences between the two approaches are the following: (a) al- 
though both approaches use a peer-to-peer architecture, in PYRAMID-S we exclude 
the clients from the peer-to-peer network. The advantage of this is that we do not 
require service requestors and providers to download and install any software; this 
results in zero deployment and maintenance cost, (b) whereas in METEOR-S ap- 
proach the client selects the domain ontology it prefers and sends the advertisement or 
query to a single Registry, our system supports publication and discovery across mul- 
tiple Registries thanks to the use of RDOs and mediators, (c) whereas in METEOR-S 
the editing of the domain classification and registry information is performed only in 
a single peer resulting in a single point of failure, in PYRAMID-S we allow editing in 
a number of peers making provision for data consistency. 

A prototype implementation of PYRAMID-S is under way. After the completion 
and the performance analysis of the current implementation, we are considering ex- 
tending PYRAMID-S functionality to support service composition whenever no direct 
matches are available to fulfill a request. Furthermore, we plan to use Content Distri- 
bution Networks (CDN) [13] to improve load balancing and performance in the 
Gateways layer of PYRAMID-S. 
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Abstract. Enterprise IT infrastructures and their interfaces are migrating toward 
a service-oriented architecture, using Web Services (WS) as a de-facto imple- 
mentation protocol. As a result, WS-generated traffic is expected to have a con- 
siderable impact on the Internet. Despite the high amount of interest in WS, 
there have been few studies regarding their characteristics. In this survey, we 
analyze publicly-accessible WS over a 9 month period. We study the evolution 
and distributions of the WS population, and message characteristics and re- 
sponse times of each WS. We also closely analyze two popular WS sites: Ama- 
zon and Google. Some of our initial results contradict common intuition. The 
number of public WS has not increased dramatically, although there are signs 
which indicate intensive ongoing activities in the WS domain. The geographic 
distribution of public WS is largely skewed. Most importantly, the sizes of WS 
responses and their variation are smaller than those of the existing Web objects. 



1 Introduction 

Enterprise IT infrastructures are currently migrating toward a service-oriented archi- 
tecture, using Web Services (WS) as a de-facto implementation protocol. By support- 
ing service-oriented and component-based application architectures, WS provide a 
distributed computing technology for revealing the business services of applications 
on the Intranet as well as on the Internet using open and standard-based XML proto- 
cols and formats. The use of standard XML-based protocols makes WS platform-, 
language-, and vendor-independent, and so an ideal protocol for a service-oriented 
architecture. In spite of the wide acceptance of WS in computing infrastructures, there 
have been few studies on WS characteristics. 

In this paper, we analyze public WS in various ways, retrieving all the WS entries 
of a UDDI [ 1 ] Business Registry (UBR). It should be noted here that we believe most 
of publicly available WS information are found in the UBR since it is proposed as a 
global registry for every type of business services. First, we study the evolution of the 
WS population and its distribution by domain and geographic location. Second, we 
develop a methodology for estimating WS message sizes. Towards this goal, we de- 
termine the distributions of several WS characteristics, such as message styles, and 
usage of complex and elementary types. Third, we examine the liveness and response 
times of the available public WS. Lastly, using our methodology, we analyze the WS 
of two popular sites - Amazon and Google - and compare the message sizes predicted 
by our methodology with the message sizes observed during interactions with the two 
sites. 
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© Springer-Verlag Berlin Heidelberg 2004 




A Survey of Public Web Services 97 



Our initial results contradict common intuition. First, the number of public WS has 
not increased dramatically, although there are certain signs which indicate that many 
intensive activities are ongoing in the WS domain. Second, the geographic distribu- 
tion of public WS is largely skewed with about three fifths of public WS located in 
US. Third, response and request message sizes are comparable, and WS response 
messages are smaller than existing Web objects. We expect these results to benefit 
WS applications and tools developers. This survey is part of an ongoing research and 
upcoming analysis results will be published on our web site [2], 

The remainder of this paper is organized as follows. Section 2 describes our meth- 
odology for data collection and for estimation of WS message sizes, and the results of 
our analysis and experiments. Section 3 applies the techniques previously developed 
to the Amazon and Google WS. Section 4 is a brief overview of the related work. 
Section 5 is dedicated to conclusion and future work. 



2 Data Collection and Analysis 

The analysis in this section is based on the data that we collected weekly, between 8 
Aug. 2003 and 7 May. 2004, from a UBR. Only a fraction of the retrieved UBR en- 
tries are relevant to Web Services, i.e., include a reference to a Web Services Descrip- 
tion Language (WSDL) file. 



2.1 Population and Geographic Distribution 

Currently, about 1200 WS are registered in a UBR. Fig. 1 summarizes the data col- 
lected during the 9 month period. The number of ‘valid’ WS - WSDL file is retriev- 
able - is substantially smaller than the total number of WS: approximately 67% of the 
WS are not valid, which is similar to the findings of a previous UDDI integrity study 
[6]. Furthermore, many of the downloaded WSDL files are incomplete. The most 
common errors are syntax errors and omission of mandatory elements. During the 9 
month interval, the number of valid WS decreases a little, which is contrary to the 
slight increase in the number of WSDL files published. Note that there is a small but 
noticeable decrease in the number of valid WS on 10 Oct. 2003 due to a server host- 
ing 54 web services becoming unavailable 1 . Finally, we found that very few organi- 
zations update their WSDL files after publication. 

The distributions of valid WS by top level domain and geographic location on 7 
Nov. 2003 are shown in Fig. 2 (a) and (b), respectively. As shown in the figure, 49% 
of WS is hosted by the .com domain and 63% of the WS are hosted in the United 
States. Our further analysis of the distribution of WS hosting sites shows that the 
portion of sites hosted by the .net domain, and in US shrink in Fig. 2 (a) and (b), 
respectively. From this fact, we infer that a larger number of WS is hosted by the 
same .net domain and/or US-resident sites. 



1 Microsoft’s .Net WS contest server (http://www.contest.eraserver.net) hosts Web Services 
which receive Microsoft's Best of the .NET Awards. 
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Fig. 1 . Web Services in UBR. 




Fig. 2. The Distribution of valid WS by (a) top level domain, (b) geographic location. 



2.2 Styles and Structures 

By design, a WSDL file includes a comprehensive description of the associated WS. 
WSDL file analysis, thus, exposes many of the WS characteristics, such as encoding 
type, message style, and number of operations for each WS. In Section 2.3, we use 
this information to estimate the size of WS request and response messages. 

By inspecting the collected WSDL files, we found that there are many more docu- 
ment-style WS than RPC-style WS; the argument about which style is better is still an 
ongoing debate. Among the 294 valid WSDL files collected on 7 Nov. 2003, 70% 
define document-style WS and 30% define RPC-style WS. All of the document-style 
WS adopt the literal encoding and all of the RPC-style WS adopt the soap encoding. 
HTTPS is used by only 4% of these services, while the others use HTTP. 89% of WS 
have less than 10 operations. Lastly, more than 78% of the WSDL files were gener- 
ated with the Microsoft toolkit. 

We also analyze the WSDL files to determine the frequency of elementary, array, 
and compound types in WS messages. We found that responses use more arrays and 
compound variables than requests do. Fig. 3 shows the distribution of elementary 
types, with array and compound types expanded into elementary types. As most WS 
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definitions do not specify array lengths, we had to select lengths for these arrays. The 
figure shows type distributions for array lengths 2 and 16. The results show that the 
string and string array types are very common. 
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Fig. 3. Frequency of Elementary Types. 



2.3 SOAP Message Size 

Characterizing the size of SOAP messages [3] is important since WS traffic is ex- 
pected to become a prevailing traffic on the Internet. In this section, we first describe 
how we estimate SOAP message sizes using the information in the WSDL files. Then, 
we explain some meaningful characteristics of the SOAP message sizes. 

A SOAP message can be divided into four parts - HTTP header, essential tag, 
namespaces, and payload. Below is the formula used to estimate the size of a SOAP 
message: 

SOAP message size = HTTP header + essential tag ( SOAP envelope tag + 
SOAP body tag) + namespaces + payload (payload tag + summation of (type 
tag + XML representation of the type) for each elementary type field in pa- 
rameters) 

In this equation, ‘essential tag’ represents the SOAP envelope and body tags, and 
‘namespaces’ represents the aggregation of all occurrences of namespace attributes in 
a message. The namespace attribute can occur in the SOAP envelope tag, SOAP body 
tag, payload tag, etc. The ‘payload’ consists of parameters and the payload tag. 

We determine the size of each message component by examining real SOAP mes- 
sages. We investigate messages of several WS, including Amazon and Google WS 
(see Section 3). We observe that there are small variations in HTTP header and essen- 
tial tag and that most messages use 5 ~ 7 namespaces. Four of these namespaces - 
SOAP envelope, XML schema, XML instance and encoding style - are present in 
most SOAP messages. 

In order to determine the payload size, we use the following methodology. First, 
we determine the size of the payload and type tags. The payload tag is used to wrap 
up a list of parameters (RPC-style) or a XML tree (document-style) and its size has a 
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small variation. The type tag is used to declare the parameter names and types, and its 
size has a small variation, as well. Second, we estimate the number of elementary type 
fields as shown in Section 2.2. Lastly, we determine the average size of the XML 
representations for the fields of each elementary type. For most numeric types, we 
assume the sizes to be as small as realistically possible. Thus, the resulting message 
size estimate is a practical lower-bound. Detailed description on average size deter- 
mination is omitted due to the space limitation. Note that string is the most frequently 
used type and its size is highly variable. Therefore, we assume that the lengths of the 
string type fields are distributed uniformly between a minimum and a maximum 
range. Similarly, we selected a small value for the lengths of arrays. 





(a) (b) 



Fig. 4. SOAP Message size when array length is 2 - (a) request (b) response. 



Fig. 4 (a) and (b) shows the CDFs of WS request and response message sizes, re- 
spectively; the minimum string size is always 5, the maximum string size is 50, 100, 
200 characters, and array length is 2. Even when maximum string size is 200, 93% of 
request messages are smaller than 2KB. In contrast, most HTTP requests are known 
to be smaller than 500bytes [7]. Fig. 4 (b) shows that 88% of response messages are 
smaller than 2KB, even when the maximum string size is 200. String size has little 
impact on small messages, as these messages use few parameters. 

We also investigate the sensitivity of the message size distribution varying the se- 
lected array length. (The results are not reported here due to space limitation.) The 
results show very little variation in the message size distributions, especially when the 
message size is small. This suggests that a small number of WS use a large number of 
arrays. 

Lastly, we compare the distributions of SOAP messages to that of existing Web 
content. For Web content, we use the model presented in [8], which screens out the 
population factor of unique files; this approach is compatible with our analysis of WS 
message sizes. Contrary to the common expectation that SOAP messages are larger 
than existing Web objects due to XML formatting, most SOAP messages are smaller 
(see Fig. 5 (a)). For instance, while 92% of SOAP messages are smaller than 2KB, 
only 45% of the existing Web objects are smaller than 2KB. 
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Fig. 5. (a) SOAP message vs. Web objects: array length is 2 and the maximum string size is 
200 characters, (b) WS and HTTP response delay. 



2.4 Liveness and Invocation Delay 

The server providing the WSDL file is typically unrelated to the server hosting the 
WS. Therefore the liveness of the WS has to be verified directly. We wrote a small 
program called Web Services Ping (WSPing), which accesses the endpoints specified 
in the WSDL files using http/https. 



<?xml version="1 .0" encoding="UTF-8"?> 

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" 
xmlns:xsd="http://www. w3. org/200 1/XMLSchema " 
xmlns:xsi= "http://www. w3. org/200 1/XMLSchema-instance "> 

<soapenv:Body> 

<Request soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> 

<dummy xsi:type="xsd:string"> This request is sent for an academic research purpose. Please send 
an e-mail to smkim@nciab.kaist.ac.kr if any problem. Thanks </dummy> 

<Request> 

</soapenv:Body> 

</soapenv:Enveiope> 



Fig. 6. WS probing message. 



WSPing sends a simple SOAP message which has a valid HTTP header and a 
SOAP envelope. The message is shown in Fig. 6. It has only one field which is a 
message to indicate that it is not a malicious attack along with our e-mail address. 
Since the message does not conform to the expected message format, the response is a 
SOAP fault: the server cannot understand our request message. If the response con- 
forms to a valid SOAP fault message format, the WS is considered alive. 

Our weekly experiments show that approximately 16% of the valid WS are down 
and that 96% of the live WS respond in two seconds or less. Fig. 5 (b) shows the CDF 
of response times for WS as well as Web servers, as measured on 13 Nov. 2003; 
measurements performed on other dates show similar results. When probed from IBM 
Watson in US and KAIST in Korea, about 85% of WS servers are alive, and about 
2-3% more Web servers are alive. Our attempts to measure ping delays do not show 
any meaningful results, as most sites block ICMP ping messages. 
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3 Case Studies 

Using the proposed methodology, we closely analyze two popular WS: Amazon and 
Google. Both WS offer the same functionality as their Web sites. 



3.1 Amazon 

Amazon provides their WS for associates, suppliers, or developers. The ‘associates’ 
program is a business model enabling 3 rd party web site operators to link their web 
sites to Amazon and earn referral fees for the sales made through their links. Amazon 
actively supports their WS: version 1.0 was released in July 2002 with basic shopping 
capabilities; version 2.0 was released in October 2002; lastly, version 3.0 was released 
in April 2003 with an expanded API for 3 rd party suppliers and shopping cart han- 
dling. In addition to the main US Amazon site, the WS API is supported for the Ama- 
zon sites in UK, Japan, and Germany. The WS Toolkit, including examples, can be 
downloaded from the Amazon WS home [4]. 

The main Amazon WS site is located in US and it is operated by Amazon itself, 
i.e., not outsourced. Their WS operations use only string types: 279 elementary 
strings, 778 one-dimensional, 702 two-dimensional, and 40 three-dimensional string 
arrays. Most of these strings and string arrays are used in response messages, as only 
179 elementary strings and 9 string arrays are used in request messages. 

Amazon WS v3.0 API has 20 operations, shown in Fig. 7. We classify the opera- 
tions according to their functionality into Product Browse operations and Shopping 
Cart operations. First-level operations are classified according to their response mes- 
sage type. These types are shown as ovals. Second-level operations are classified 
according to request message type. As a result, operations in the same leaf node have 
the same request and response message types. HTTP and WS response delays are 327 
and 502msec when measured from IBM Watson, and 501 and 510msec from KAIST. 




Fig. 7. Operation Tree of Amazon Web Services. 

Fig. 8 (a) and (b) show the message sizes, both real and estimated, for requests and 
responses, respectively. We assume array length of 2 or 16. Note that browse opera- 
tions have two kinds of responses: lite and heavy. A lite response delivers the sum- 
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mary of the selected items, while a heavy response delivers all the available informa- 
tion. The fixed size components are identical but the payload varies widely. 






(a) (b) 

Fig. 8. Amazon WS Message Size (a) request (b) response. 



Fig. 8 (a) shows that our estimation of request sizes is accurate, especially when 
the array length is 2. The only large gaps, when array length is 16, are due to the fact 
that the addcart and modifycart operations use string arrays. Fig. 8 (b) shows the re- 
sponse message sizes; estimations are less accurate. Note that only the sizes of heavy 
type messages are estimated. The pattern of lines is almost identical and the line for 
heavy-response is between estimated lines. To improve accuracy, application specific 
information is needed. 



3.2 Google 

Google provides a WS API to their Web search engine in order for developers to 
embed Google search functions into their programs [5]. The Google WS API was 
launched in April 2002 and is still at the beta version. The Google API has only three 
operations: doGetCachedPage, doSpellingSuggestion, and doGoogleSearch. These 
operations use 14 elementary strings, 11 one-dimensional string arrays, 5 Booleans, 
and 5 Integers. HTTP and WS response delays are 292 and 329msec when measured 
from IBM Watson, and 841 and 1046msec from KAIST. 

Fig. 9 shows the message sizes of Google WS. We assume maximum string size of 
50 characters and array length of 2 or 16. The figure shows that our estimation of 
message sizes is accurate except for the response message size of doGetCachedPage. 
In this case, as Google returns a cached Web page as a single parameter of byte[] 
type, array length should be much larger than 16. 



4 Related Work 

To the best of our knowledge, this is the first survey of public WS. Although there 
have been many studies on Web evolution [14,15] and Web site evaluation [16], they 
do not study WS sites. 
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— a — request 

------- response 

— • — estimated request 
(array 2) 

— * — estimated response 
(array 2) 

— * — estimated request 
(array 16) 

— • — estimated response 
(array 16) 

getcachedpage search spellingsuggestion 

Fig. 9 . Google WS message size. 

Mapping and analysis of the Internet geography have been studied in detail [9]. 
These studies focus on the geographic location of Internet components such as end 
nodes and routers. Our investigation considers a geographic distribution of WS end- 
points, not just IP-level internet nodes. 

WS portals [10,11,12,13] provide information about useful WS, including cate- 
gory, rate, price, and service explanation. Most of this information targets WS con- 
sumers. We investigate the evolution, internal structures, and message characteristics 
to improve the understanding of WS technology. 




5 Conclusion 

In this survey, we study several aspects of public WS using the information that we 
collected weekly over the past 9 months from an UBR. First, we determine the WS 
population and its distributions. Second, we develop a methodology for estimating 
WS message sizes. Towards this goal, we determine the distributions of several WS 
characteristics, such as message styles, and usage of complex and elementary types. 
Third, we examine the liveness and response time of each WS. Lastly, using our 
methodology, we analyze the WS of two popular sites in detail. 

Our initial results show that the number of public WS does not increase dramati- 
cally and that about three fifths of the current WS population is based in US. In addi- 
tion, our results indicate that there are substantial differences between the sizes of 
request/response messages for public WS and current Web traffic. Lastly, string type 
is much more frequently used than the other types. 

We expect these results to be beneficial in various ways. For instance, WS tool de- 
velopers may optimize their products for the preferred message style and frequently 
used variable types in SOAP messages. In addition, the proposed message size esti- 
mation methodology helps WS server or network administrator to perform resource 
planning and configuration easily without analysis on actual usage log. 

We plan to extend our survey by collecting more WSDL information from other 
sources. We also plan to refine our methodology for WSDL analysis as well as mes- 
sage size estimation. For instance, we plan to use the semantics of the WS operations 
to estimate string and arrays lengths. This survey on public WS is part of an ongoing 
project and upcoming analysis results will be published on our web site [2]. 
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Abstract. Existing electronic negotiation systems (ENSs) typically implement a 
single, fixed negotiation protocol, which restricts their use to negotiation prob- 
lems that were anticipated and established a priori by the system’s designers. 
The single-protocol restriction limits ENSs’ applicability in experiments and in 
many real-life negotiation situations. ENSs that allow for the use of different 
protocols also allow for the customization to users" needs and abilities. We pre- 
sent theoretical foundations for the design of flexible and highly customizable 
protocol-driven ENSs. 



1 Introduction 

The term e-negotiation systems has been used to describe software that employs 
Internet technologies, is deployed on the World Wide Web, and capable of support- 
ing, aiding or replacing one or more negotiators, mediators or facilitators [ 1], A num- 
ber of ENSs have been designed, implemented and applied to various negotiation 
problems. Some systems facilitate negotiation of documents and their joint prepara- 
tion, e.g., contract negotiations [2], others use email, chat and streaming video soft- 
ware [3]. An overview of different ENSs can be found in Shim and Hsiao [4] and 
Neumann et al. [5]. 

With few exceptions - notably SilkRoad [6] and INSS [7] - existing ENSs imple- 
ment only one fixed negotiation protocol [8]. This restricts the use of ENSs to types of 
problems and interactions that were assumed and established a priori by their design- 
ers. This, in turn, imposes limitation on the behavioural research of the ENSs’ use and 
their efficiency and efficacy, on the ENSs’ applicability to support evolving negotia- 
tions, and those conducted by users who have different needs, cognitive abilities, and 
cultural and professional backgrounds. 

Ongoing behavioural research on ENS focuses on (i) technology adoption by nego- 
tiators, and (ii) the impact of different systems on the negotiation process and negoti- 
ated outcomes [9, 10]. Both research directions utilize experimental and empirical 
methodologies. From this perspective (in particular in experimental studies of ENSs’ 
use and adoption), the assessment of the impact of different system features on the 
process and outcomes of negotiations requires the use of systems, whose differences 
and similarities can be easily controlled by the researcher. From a negotiator’s point 
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of view, the limitation to a single fixed protocol restricts the use of a particular ENS to 
the supported class of negotiation problems, which may not include their problem at 
hand. If, on the other hand, ENSs implement negotiation protocols, which apply to a 
large class of negotiation problems, i.e, are very general, they impose significant 
cognitive and informational demands on the users who need to make decisions about 
the selection of tools and features. Negotiators who use a system to negotiate need to 
concentrate on the problem and process, and make decision about the concessions 
rather than compare different tools and decide about system features. It is thus advan- 
tageous that: (1) a protocol be constructed for the negotiators based on their character- 
istics and the negotiation problem and context, or (2) the negotiators decide on a ne- 
gotiation agenda, which sets a particular formal protocol. 

The purpose of this paper is to present the theoretical foundations of negotiation 
protocols 1 . These protocols can be implemented in a software platform and then used 
for the construction of various ENSs. The remainder of this contribution is organized 
as follows. Section 2 briefly reviews vital elements of a negotiation methodology. 
Section 3 introduces the theoretical foundations of negotiation protocols and their 
properties, and Section 4 presents ongoing and future work. 

2 Negotiation Methodology 

Negotiation methodology describes the methods, procedures, and techniques used to 
collect and analyze information used in negotiation, the process of communication, 
exchange of offers and concessions, and arrival at an agreement or deadlock. It is 
important that these methods and techniques match the negotiator’s capabilities, com- 
plement each other, do not produce contradictory information and - when used - 
contribute to the negotiation effectiveness. 



2.1 Negotiation Process and Activities 

The use of a methodology has been advocated by negotiation experts, but this advice 
is often neglected in unstructured negotiations (e.g. face-to-face or via e-mail). One of 
the important contributions of an ENS is to provide a methodology, which matches the 
negotiators’ requirements and is appropriate to their problem. The use of a methodol- 
ogy in an ENS is also required for the tractability of the process and its ease of use. 

For the purpose of this work, we consider only two key components of the negotia- 
tion methodology: (1) the negotiation process model, and (2) the negotiation protocol. 
The process model provides a framework for negotiations; it organizes the activities 
undertaken by negotiators by grouping them into negotiation phases and by assigning 
different activities to each phase. It serves as a starting point for the software design 
and draws its significance from imposing a methodologically sound approach to nego- 
tiators [12]. The protocol is a formal model, often represented by a set of rules, which 
govern software processing, decision-making and communication tasks, and imposes 
restrictions on activities through the specification of permissible inputs and actions 
[13, 14]. Negotiation protocols are further discussed in the next section. To our 



1 This paper is a revised and short version of Kersten, G.E., S. Strecker, and K.P. Law (2004). 
Protocols for Electronic Negotiation Systems: Theoretical Foundations and Design Issues, 
InterNeg Working Paper 06/04: Ottawa, Canada. 1-16. 
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knowledge, there are no behavioural studies on e-negotiations and, therefore, no proc- 
ess model specific to e-negotiation has been developed. For the purpose of designing 
and implementing an ENS, we use a five-phase model based on Gulliver’s eight-phase 
model [15], which allows for the consideration of a wide range of negotiations, in- 
cluding those supported by ENSs. The five phases are planning, agenda setting, ex- 
changing offers and arguments, reaching an agreement and concluding a negotiation. 
Each negotiation phase has its own purpose and set of activities, which are concrete 
actions undertaken by each negotiator. The purpose of the different negotiation phases 
is to provide the participants with a framework and rationale for activities conducted 
in each phase. The consideration of phases helps to specify negotiation activities un- 
dertaken and the relationships among them. 

The negotiation process model provides a framework, but it does not impose any 
restrictions on the negotiators concerning the sequencing of phases. In any given 
phase, the negotiators may revisit previous phases and then return to initial phase. 
Moreover, it often occurs in real-life negotiation that negotiators skip or ignore one or 
more phases. Although negotiation experts suggest that all phases should be consid- 
ered, we leave this issue to the protocol designer as there may be specific situation, in 
which one or more phases should be bypassed. 



2.2 Negotiation Protocols and Activity Types 

Any negotiation supported by an ENS requires that the software designers precisely 
define the activities and their sequence using a negotiation protocol [10, 13]. The 
negotiation protocol defines the activities that are permissible in every state of the 
negotiation, their sequence as well as input and output requirements. The key con- 
cepts used to define the activities and to specify their sequencing are presented in 
Figure 1. 

Behavioural theory posits that activities depend on the negotiators’ characteristics 
and the negotiation context (e.g. power distribution, and the relative importance of 
outcomes). These characteristics determine the negotiators’ approaches, their strate- 
gies and tactics leading to the selection of specific activities from the negotiation 
phases. Behavioural research cannot provide sufficiently precise insights regarding 
sequencing of activities within each negotiation phase. 

This is because of the number of possible combinations of the negotiator’s 
characteristics, interdependencies between characteristics of the negotiators, 
dependence of the negotiators’ behaviour on external factors (e.g. the relationship 
with other stakeholders), and the complexity of the problem and process. With the 
exception of well-defined and highly structured negotiations, such as those taking 
place in procurement of standardized goods, the negotiators cannot follow a strict set 
of rules defining the activity’s sequence. 

The above mentioned complexities introduce the requirement for providing the ne- 
gotiators with some degrees of freedom in the selection of activities. During the proc- 
ess, the negotiators may wish to review the problem, modify their preferences, add or 
remove issues etc, which imposes the requirement of some activities to be optional 
and/or exchangeable for other activities. Also, the negotiators may be forced to under- 
take certain activities in order to move to the next activity. For example, they should 
learn about the negotiation problem, consider their own objectives and preferences 
and evaluate the counterpart’s offer before making their own offers. To accommodate 
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these requirements, we distinguish between mandatory activities and optional activi- 
ties (see Figure 1). 



Theory-based specification 



Protocol-based specification 



Negotiation 




theories. 




approaches 




and models 





(-* Process model 






I 



Strategies and 
tactics 

i 
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Activities 



Fig. 1 . Theory- and protocol-based activity specification. 



The distinction between mandatory and optional activities is necessarily context- 
dependent; the same activity may be mandatory in one state of the negotiation and 
optional in another. For example, when the user enters the system for the first time, he 
is required to learn about the negotiation problem; at this time this activity is manda- 
tory. When he logs in the second and subsequent times, learning about the problem 
should not be a mandatory but an optional activity. It is the negotiation protocol that, 
based on the process model and the assumptions of the protocol designer, categorizes 
some activities as mandatory or as optional, and modifies this categorization as the 
negotiation progresses. The assumptions underlying a specific protocol may reflect 
the negotiators characteristics (e.g. culture and profession), the type of negotiations 
(e.g., distributive, integrative, and mixed), and the complexity of the negotiation prob- 
lem (e.g. one or more issues). In the research environment, these assumptions may 
also reflect the needs of the researcher studying the users’ behaviour and the system’s 
efficacy. 

Different negotiators and negotiation situations require that different protocols be 
used. The protocols may differ in the sequencing of the same set of activities. Con- 
versely, the same set of activities may be (re-)used in the construction of many differ- 
ent protocols. Negotiation strategies and tactics also may require the use of different 
protocols. Moreover, different types of negotiations, e.g. single or multiple issues, and 
different roles of the negotiators, e.g. buyer or seller, require different protocols. An- 
other need for different negotiation protocols derives from the requirements and de- 
mands that different stakeholders have regarding their use of an ENS. To meet the 
requirements of negotiators and researchers, it is essential to equip an ENS with the 
flexibility to carry out several different protocols and to provide the user or researcher 
with the possibility of designing new negotiation protocols. 



2.3 Process Model and Negotiation States 

The framework provided by the process model is implemented in the negotiation 
protocol, which is represented by a sequence of activities and rules imposed on the 
execution of the sequence. Additionally, the execution of a protocol depends on the 
context of a negotiation, or more precisely, on the current state of a negotiation a user 
is currently involved in and on the user’s earlier actions in that negotiation. The proc- 
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ess model reflects the progression of a negotiation as it tracks the completion of 
phases and activities. An example of the process model, and its phases and activities 
is given in Table 1. We use these phases and activities to illustrate, in Section 3, for- 
mal protocol construction and manipulation. 



Table 1. Example of the process model, activities and states. 



Negotiation phase and activity 


State 


Abbr. 


1. Planning 

- Negotiation problem 


Negotiation case 


NC 


2. Agenda setting 
- Preferences and rating 


Utility construction 


uc 


- Assessment of alternatives 


Alternative construction 


AC 


3. Exchanging offers and arguments 
- Offer and/or message construction 


Offer message 


OM 


- Counter-offer assessment 


Counterpart's offer 


CO 


4. Reaching agreement 
- Agreement 


Agreement reached 


AR 




Agreement assessment 


AS 


- Closing negotiation 


End 


EN 


5. Concluding negotiation 
- Agreement improvement 


Agreement improvement 


Al 


- Offer and/or message 


Offer message 


OM 


- Counter-offer assessment 


Counterpart’s offer 


CO 


- Closing negotiation 


End 


EN 



Each negotiation activity is associated with an ENS state (see Table 1); however the 
reverse is not true: The system may be in a state that does not correspond to any nego- 
tiation activity. For instance, state AS involves agreement efficiency analysis and does 
not correspond to any activity. The difference between a negotiation activity and an 
ENS state is that the former describes a user action, while the latter denotes a user 
and/or a system action. 

3 Negotiation Protocols 

Every ENS implements a negotiation protocol - even though some system designers 
do not specify the protocol explicitly - the protocol can be derived from the required 
and possible interactions between the negotiators and the system. It is sensible to 
formulate the negotiation protocol explicitly, because it specifies the users’ interac- 
tions and thus the users need to determine if the system conforms to their require- 
ments. In addition to the interaction transparency introduced by explicit protocols, it 
also allows for mapping protocols onto negotiation processes. Furthermore, it is also 
possible to assess the protocols’ underlying assumptions. Various formalisms have 
been applied to represent negotiation protocols, e.g. Petri nets [16] and state chart 
diagrams [17], Our approach is based on set theory mainly for its flexibility and read- 
ability. 

3.1 Preliminaries and Conditions 

Following the distinction between mandatory and optional activities (see Figure 1), 
we distinguish two types of ENS states: mandatory or optional. Let: 
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S = {jj, s N ] be the set of all possible states; 

M be the set of mandatory states (M cz S); 

O be the set of optional states ( O c S); 

s g S be the first state of the protocol; and 

.s'end g S be the last state of the protocol; it is the protocol termination state. 

We assume that Sj = s and s N = ,s cnd . Every state is associated with at most one 
mandatory state, which defines a partial protocol sequence, i.e. a sequence is a pair of 
two states: 

Sj — > Sj, Sj ± Sj with i, ye/, (1) 

where ,v ; e 5; Sj e M; and |/| = N is the number of states. 

Using the states given in Table 1, we can formulate several sequences, including 
the following two sequences: NC — > UC, UC — > AC. State UC is mandatory for NC', 
the user can move to state AC only after completing activity associated with UC. 
Similarly, AC is mandatory for UC. Optional states allow the user (or system) con- 
ducting activities within a sequence prior to moving to the next sequence. 

Sj — » Sj opt Of, with Sj ^ Sj, Sj e S, Sj g M, O t c O, (2) 

where opt is the operator of state association and 0 ; is the set of optional states associ- 
ated with state j.. 

Formula (2) is interpreted as follows: The user who is in state $. can visit states s t g 
Oj multiple times and return from these states to s p but he cannot move to any state s k 
(s k ^ Sj, Sj e S\ Oj), before he visits state s- e M. For example, the user who assesses 
alternatives (AC) can return to the description of the negotiation problem ( NC) and 
revise his preferences (UC), but he cannot move to any other state listed in Table 1, 
unless he formulates an offer, that is: AC —> OM opt {NC, UC}. 

A state may have a null state associated as mandatory state as long as the set of op- 
tional states is non-empty. The resulting sequence with a mandatory null state is de- 
noted as: 

Sj — > 0 opt Oj with Sj e S, M = 0, Oj a O, (3) 

where 0 is the null state. Similarly, a state may have a mandatory state with an empty 
set of optional states (Oj = 0): 

Sj — > Sj opt 0 with Sj g M. (4) 

Based upon these preliminaries, negotiation protocols are required to meet certain 
general conditions. 

Condition 1: If state has null mandatory state, then at least one optional states has to 
be associated with j., i.e. 

Sj — > 0 opt Oj => Oj -A 0. (5) 

Condition 1 is required in order for the user to be able to move from the state with 
which no mandatory state is associated to one or more optional states. Moves between 
optional states are not considered sequences. Assume that set Oj in (5) has three ele- 
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ments CL = { Sj, .v ; , s k }. The user can move from any optional state to any other op- 
tional state, e.g., he can move from s. to s, to j.. One implication is that all optional 
states in (5) are elements of CL. The second implication of (5) is that to move from s, 
or one of its optional states to a state, which is not an element of CL. is possible only if 
there is a state in CL, which is a part of another sequence of type (2) or (3). This re- 
quirement is formulated in the following condition. 

Condition 2: With every state ,v ; , which has a null mandatory state, at least one op- 
tional state has to be associated, which is part of another sequence, that is: 

V s t : Sj — > 0 opt CL : 3 ,v ; , s, e CL and (s, — > s, s- e M or s t = s end ). 

This condition assures that the user can move from any state to either a state with 
which a mandatory state is associated or to the termination state s end . Condition 1 and 
2 do not assure that every state can be accessed by the user; this is achieved if the 
following condition is met. 

Condition 3: With the exception of the starting state (s start ), every state s j e S is man- 
datory and/or optional, i.e. 

v s i> s i * W s i e s : s t e M u O. 

Condition 4: A mandatory state may appear only in one sequence of the protocol, 

V Sj, s i e S, S' — > Sj => — i 3 s t — » s ■, s t s t , Sj e M. 

The above four conditions define the required characteristics of every protocol. 

Definition 1: q(i,k ) is the list of k sequences that begin with state s i and end with the 
mandatory state s i+k , such that: 

1. Sj is an optional state for ,v ; _ , ; 

2. The mandatory state for s i+k is the null state. 

3. No state, other than s i+k , in q(i,k) has mandatory null state; and 

4. With the exception of s j+k every mandatory state is the first state in one other se- 
quence in q(i,k). 

The sequence q{i,k) is: 

q(i,k) : s ( -> .v i+l , s i+1 -> s i+2 , S' +k l -4 s i+k a s, t M a s i+k -» 0. (6) 

The list q(i,k) can be represented as a graph starting at state y and ending at s i+k . 
Every state starting a sequence has a mandatory state which starts another sequence, 
with the exception of the last state s k . The sequences in the list may or may not have 
associated with optional states (see Figure 2). Let: 

J be the index set of lists of sequences; q(ij,kj ) denotes the /-th list in the protocol; 

Q = {q(ij,kj), (j e J)} is the set of all lists of sequences of the type given by (6) in 
the protocol; 

P = { Sj — > 0 opt O', i e /} be the set of all sequences in which the mandatory state is 
the null state; 
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S j (S [ a S) be the set of states that are elements of the mandatory and optional sets 
associated with ,v ; and the states preceding s ( , i.e., = {s start , s 2 , ..., s ( ; 0 start , 0 2 , ..., 

O-,); S j+ (S j+ d S) the set of states that are elements of the mandatory and optional sets 
associated with ,y j+| and states following s !+1 , i.e. s i+2 , . .., s end . 

Note that S j+ n SI is not necessarily an empty set because some states may appear 
in more than one optional sets both preceding, including and following ,v-. 

Definition 2: Negotiation protocol p is the 5-tuple: 

P = (S, O, M, P, Q) (7) 

3.2 Protocol Completeness and Modifications 

Given the definition of a negotiation protocol p, we are able to establish a complete- 
ness theorem of negotiation protocols and proof the completeness of a protocol in the 
sense that all states of the protocol can be visited. The theorem and proof are available 
in the companion working paper [11]. The discussion of different protocol states and 
their relationships in Section 3.1 does not take into account the dynamics of the nego- 
tiation process. We formulated Conditions 1-4, which specify protocol properties 
required to introduce, in particular, the concept of a complete protocol. This is not to 
say that there may not be situations in which the protocol designer may want to vio- 
late one or more of these properties. One such example is that termination state ^ end 
occurs only once in the protocol. For practical reasons this condition may be purpose- 
fully violated, so that the user has an option to terminate the negotiation at every state 
rather than be forced to follow the protocol until he reaches state s end . 

The distinction between mandatory and optional states partially takes into account 
context-dependency; it only allows to access different optional states before moving 
to a mandatory state. Protocol p defined with (7) may be used in highly structured 
protocols, which follow the rules implemented in p. The rules governing the moves 
between states are static; they do not depend on the states visited. A stronger require- 
ment reflecting protocol context-dependency is to allow for the states’ rearrangement 
during the negotiation. In many negotiations, the permissible states depend on states 
previously visited. The dynamics of the negotiation process are reflected in context- 
dependent modifications of the negotiation protocol at run-time, which means that 
some mandatory states may became optional, some optional states may be added and 
others removed depending on the user and system actions. 

3.3 Intervening States 

The characteristic that distinguishes negotiations from individual decision making is 
exchange of information between the negotiators (e.g., offers, messages and offer 
acceptance). Information sent by one negotiator affects his counterpart’s activities. 
Therefore, the system has to display this information at the earliest possible time and 
irrespectively of the state he wants to visit. The states which contain and process in- 
formation sent by the counterpart are called intervening states. 

In face-to-face negotiations, the negotiators may exchange messages even during 
the planning phase. The synchronous aspect of these negotiations causes that they 
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rarely formulate offers before both sides are ready to negotiate. Asynchronicity of 
negotiations conducted via an ENS causes that one party may learn about the problem 
and be ready to exchange offers and messages, while the other party does not yet 
know the negotiation problem. This may require that the user be moved from some 
states to an intervening state, but not from other states. For example, if the user is in 
state NC and his counterpart makes an offer, this offer should not be displayed to the 
user prior to his specification of preferences in state UC. We therefore associate with 
every intervening state a set of permissible states ; these are the states from which the 
user can be moved to the intervening states. 

4 Summary and Future Work 

In this paper, we lay out theoretical foundations for negotiation protocols based on a 
negotiation methodology derived from behavioural research. The foundations facili- 
tate the design and implementation of negotiation protocols and allow for the con- 
struction of ENSs based on these protocols. The construction of negotiation protocols 
may be a highly complex task. The theory in Section 3 will allow for the implementa- 
tion of a software tool that supports protocol designers and automatically verifies, if 
the particular protocol meets the formulated conditions. 

We are currently designing and implementing the software platform Invite, which 
will serve as a run-time environment for multi-protocol ENSs on the one hand and as a 
host for software tools for protocol design and verification. Our approach strives to 
achieve a level of genericity for the platform that enables the construction of multi- 
protocol ENSs from existing parts and thus allows for reusability of predefined com- 
ponents. The software platform will execute different negotiation protocols in differ- 
ent ENSs and thus will simplify the adoption of e-negotiations in real-world and in 
research environments. Our future work includes the implementation of components 
for several bilateral negotiation protocols, and a respective protocol designer support 
tool. We also plan to extend the Invite platform to multi-bilateral and multilateral e- 
negotiations. 
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Abstract. One theoretical approach to designing and constructing complex 
market structures is the concept of Cascading Dynamic Market Models 
(cDMMs). CDMMs allow the configuration and combination of multiple mar- 
ket models. MetaMarkets is presented here as an implementation concept for 
complex market structures. MetaMarkets form a concatenation of a set of mar- 
kets and rules that represent relations between single market structures and their 
environment. Various market model combinations and their representation are 
discussed, and a communication method with the environment presented. 
MetaMarkets lead to a layered software architecture that we briefly depict. 



1 Introduction 

Electronic marketplaces facilitate the exchange of goods, services, information, and 
payments and create economic value for buyers, sellers, market intermediaries, and 
for society [2]. Bakos states that the main functions of markets are matching buyers 
and sellers, facilitating market transactions, and providing an institutional infrastruc- 
ture enabling the efficient functioning of the market. Thus, market design focuses on 
the design of “efficient markets” providing “precise and accurate information to all 
participants and giving them the ability to identify and exploit all advantageous 
trades” [1], The institutional rules that make the electronic market work, and the proc- 
ess that installs these rules, have to be analysed on the basis of market participants’ 
requirements, economic efficiency, and social justice. 

Market participants have various - and to some extent - contradictory requirements. 
Thus, it would appear desirable to have many electronic markets, each fulfilling the 
participants’ requirements concerning the traded goods and the trading rules. This 
would lead to many different markets, and consequently to a split of liquidity between 
all markets. As this dilemma of liquidity versus adaptability cannot be resolved in 
conventional electronic trading systems, a new market concept is needed in order to 
fulfil investors’ requirements and to avoid the splitting of marketplace liquidity. One 
way would be to combine markets, which not only implies the integration of markets 
based on identical market models, but also the combination of different market mod- 
els. Alternatively, the integration of markets leads to the synchronous existence of 
orders within two or more markets that, for their part, raise economic and technical 
questions concerning the feasibility and reasonability of such concepts. 

This paper does not discuss the economic reasonability of market integration with 
reference to the corresponding literature [7]. Instead it demonstrates a novel compo- 
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nent based concept for the implementation of market integration. In this concept, the 
market structures are concateneted with the information for their management. From 
this concatenation, questions arise such as: "Which kind of combinations of market 
models are possible?"; "How can MetaMarkets communicate with their environ- 
ment?", and "What does the software architecture for MetaMarkets look like?". Be- 
cause this concept joins market structures that belong together with their management 
information we call it MetaMarket. 

The remainder of this paper is organized as follows. Section 2 introduces Cascad- 
ing Dynamic Market Models as a solution for the liquidity versus adaptability di- 
lemma mentioned above. Section 3 briefly presents components that are used for 
component based market modelling. Section 4 introduces the MetaMarket concept as 
an extension of component based modelling and presents some key issues surround- 
ing it. The implementation of MetaMarkets is then discussed in Section 5. Finally, the 
paper concludes in Section 6 with remarks and discussion about our future research. 



2 Cascading Dynamic Market Models (cDMMs) 

The individual preferences and requirements of market participants concerning the 
trading rules raise a challenge for the market designer because their fulfillment ap- 
pears to be impossible using traditional concepts of market models. This section fo- 
cuses on the market microstructure (one perspective of the market engineering con- 
cept) and proposes a complex market structure (in this paper we use the terms 
“market structure” and “market model” synonymously) mapping the requirements of 
the traded product as well as the needs and demands of the market participants. These 
needs and demands require a market design that induces market efficiency and, in 
particular, provides precise and accurate information to all participants, efficient and 
reliable communication with one or several markets simultaneously, safe and trust- 
worthy exchanges and ensures the correctness and efficient computation of market 
decisions [1]. Therefore, we propose a new market concept that provides the integra- 
tion of markets within one market model - that is to say, multiple market models 
combined on one trading platform fulfilling the requirements listed above. 

[3] introduce the concept of “ Dynamic Market Models (DMMs)“. Market partici- 
pants themselves are given the opportunity to choose market microstructures’ charac- 
teristics according to their preferences in a DMM. This idea of dynamic market mod- 
els fulfils the postulated characteristic of more individual market design, but does not 
provide much more flexibility than traditional market models: the market participant 
just chooses one set of market parameters and therefore one market he wants to place 
the order in. 

[7] extend the concept of DMM to “ cascading Dynamic Market Models 
(cDMMs)”, considering the integration of single market models within one order 
book. We suggest a more comprehensive interpretation of this concept of cDMMs. 
The cascading concept supports the configuration and combination of multiple market 
models and is, thus, an extension of the concepts presented by [3] as well as [7], For a 
comprehensive definition and description of cDMMs, the following two perspectives 
must be considered: (1) the market designer’s view and (2) the order’s view. 

Form the market designer’s point of view, cDMMs allow the market designer to 
determine the market structure, that is the parameters of the market mechanism and 
the trading rules, as well as a combination of multiple market models within the trad- 
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ing platform. This combination can either be parallel (parallel market models) or 
sequential (sequence of market models). From the order’s perspective, cDMMs allow 
the market participant not only to choose more than one market to place the order in 
simultaneously, but also to define preferences for the sequence of markets the order 
has to pass through. 



market view 



order view 



M6 > 



M5> 



M3 



E1M1112 



Ml 



o- 



order A 

O 1 

order B 



Ml 






M3 



M6 



Ml 



M2 



M5 



Fig. 1. Cascading Dynamic Market Model - market’s and order's perspective. 



This is depicted in Figure 1. As shown in the market view, six markets exist, Ml to 
M6, each market individually designed, e.g. by different trading rules and matching 
algorithms, and combined either in parallel or sequentially to a cDMM. The orders A 
and B, illustrated in the order view, choose a sequence of markets to be traded in. 
Order A has the preference of first being traded in market M 1 , then, if not executed in 
this market, moving on simultaneously to the parallel markets M2 and M3, and fi- 
nally, being traded in market M6. A second example is given for order B. Order B 
first goes into market M4. If not executed here, order B exists simultaneously in Ml 
and M2 and afterwards, if not executed, enters market M5. 

In a first step, we have suggest the idea of cDMMs as an integration of multiple 
market models within one trading platform. The market microstructure, e.g. the trad- 
ing rules and matching algorithms, has to be defined. As we postulate a generic ap- 
proach to define the market structure depending on the product specifications and the 
demands of the market participants, we propose, in a second step, a component based 
approach. 



3 Components 

The central questions for the implementation of electronic markets are 1) which trad- 
ing rules are needed? and 2) how can they be implemented and finally composed to a 
concrete market model? In [6] a component based approach for the definition of mar- 
ket models is given. The authors define components as rules and algorithms that build 
the fundamental criteria for the market structure and for the definition of its character- 
istics in following way: 

Definition: Component 

Given a set of rules R = jR I ,R 7 ,....,R k j with R j ,/e k mutual independent 

rules, and a set of algorithms A = f A 1 ,A 2 ,....,A I j with A-, j& (l,...,lj, l mutual inde- 
pendent algorithms. Then C is a component defined as a set of rules R and a set of 
algorithms A that is C = (R, A). 
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For the concatenation of these components to new components and finally to new 
market structures, the authors define a logic composition operation for two or more 
components as follows: 

Definition: Compositon 

A composition is a logic operation of two or more components. 

Given two components C 1 and C2, each component defined by a rule and an algo- 
rithm, a new component is achieved by the logical combination of Cj and C 2 , that is C 
= C 1 o C, = C 2 o Cj. In this case, C 2 and C 2 are subcomponents of C. 

Having determined components and the composition of components, a definition 
for a component based market structure can be given in following way: 

Definition: Component based market structure 

Given a set of rules R, a set of algorithms A, and n components C ; - = /R 1 , A'} for i = 
l,...,n with R 1 <z R a subset of R and A' ci A a subset of A we define a market structure 
based on components as a composition of n Components, that is a component based 
market structure 

M=C 1 oC 2 o...oC n = (R 1 , A 1 } o / R 2 , A 2 ! o .... o (R n . A'f. 

These definitions propose an abstract way to support the design and configuration 
of complex market structures in an easy manner. According to these definitions, a 
market structure can be built up from components using composition. 

These criteria, or parameters, have been discussed for example by Wurman et al. 
[9], by Lomuscio et al. [5], by Strobel and Weinhard [8]. In particular, the Montreal 
Taxonomy [8] gives a comprehensive overview of the criteria, parameters, rules, and 
algorithms necessary to define a market structure. These criteria are independent and 
orthogonal and form an n-dimensional criteria space, where “n” is the number of all 
criteria necessary to define a market structure. 

Following sections present an approach analogous to the component based compo- 
sition of market models for the definition and implementation of complex market 
structures. 

4 MetaMarket 

This section focuses various aspects of the implementation and realization of multiple 
markets. Thus, the concept of MetaMarkets is presented as one approach to the im- 
plementation of cDMMs. 

CDMMs can be generated in two ways: by a market designer, or by an investor. In 
the first case, the market designer configures each market model individually and 
creates a complex market structure by combining two or more (individually config- 
ured) market models to one logical unit. This was mentioned above as the “market 
designer’s view”. In this case, the market designer joins two or more markets running 
simultaneously or sequentially in such a unit. Although the internal structure of each 
logical unit (that is the setting of the market structures’ parameters of each market 
model and the combination of these market models) is visible to the market partici- 
pants, each unit can be treated as one single market. Thus, it is characteristic for 
cDMMs that the market designer fixes their internal structure as well as additional 
rules. 
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In the second case, an investor can generate cDMMs ad hoc (on the fly). The in- 
vestor inserts the information about the markets into the order which the order should 
enter simultaneously or sequentially. Focussing on the market models the order has to 
pass through, we get a cDMM that exists only from the order’s point of view. This 
case is mentioned above as the “order’s view”. Accordingly, each order potentially 
contains a cDMM of its own. The implications of both ways for the definition of a 
cDMM are discussed below. 



order O 

O — 



cDMM 



cDMM 



cDMM 




Orderbook of Ml R 

Fig. 2 . Order books in cDMMs. 




As mentioned above, our concept of cDMMs is an extension of Neumann et al. 
(2002). The authors consider the integration of single market models within one order 
book. Due to the necessity to integrate any kind of market structures, the integration 
cannot be based on one order book. Consider the example of mutual different match- 
ing rules: each matching rule requires a separate and differently organized order book. 
This problem is particularly obvious from the order’s point of view: each market 
participant is able to define a virtual cDMM with an individual order book for each 
order. Referring to this, it seems impossible to define a common order book. Thus, we 
propose a more general concept of market integration. In our approach, each market 
has its own order book and every single order may register itself in the multiple order 
books synchronously, according to the rules that the investor has defined for the or- 
der. Figure 2 illustrates the situation described above. The order, O, that is first enter- 
ing a cDMM, registers itself in (the order book of) M 1 and can be matched in M 1 . If 
no matching occurs, O cancels its registration in Ml and registers synchronously in 
M2 and M3. 




Fig. 3. Markets and internal rules of a MetaMarket. 



In this context, new problems arise: the synchronization of accesses and deadlocks. 
These cases are deeply discussed in Czernohous et al. (2003) [4] and do not fall 
within the ambit of this paper. 

Analogous to the definition of market structures by rules and algorithms, a market 
designer defines cDMMs as a combination of multiple markets. This combination of 
the markets is based on rules between these markets. The concept of MetaMarkets we 
propose is one way to manage the combination of multiple markets and the rules 
between the markets. 
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Definition: MetaMarket 

A MetaMarket is defined as a set of markets and rules representing relations between 
markets (intern rules R 1 ) and their environment (extern rules R c ). Let MM be a 
MetaMarket, M:={M p M 2 , ..., MJ a set of markets, and R:= fR p RJ a set of 
rules. Then we can define MM as a composition of markets and rules with 

MM := MjO M 2 o ... o M k o Rj o R 7 o... o R n , k, n G N, R- £ { R' x , R C J, x, y € N. 

Note that defining the MetaMarkt as a composition we consider both markets and 
rules as components that are combined to complex market structures by the composi- 
tion. Internal rules of the MetaMarket define, for example, the sequence of the mar- 
kets and their validity over time, e.g. discrete time points for starting and stopping 
markets. External rules define, for example, the sequence of markets an order has to 
pass through within a MetaMarket. Figure 3 sketches the setting of a MetaMarket 
with two markets Mj and M 2 and rule set R. 

As already mentioned, rules are necessary to combine multiple markets to a 
cDMM as a complex market structure (market designer’s view) or to define the se- 
quence of markets an order has to pass through (order’s view). The markets and the 
rules are managed together and coordinated by MetaMarkets. Nevertheless, each 
market contains trading rules and algorithms that are defined by market structure 
parameters (cf. Section 3). Such rules can either be time based (e.g. fixed or relative 
starting and stopping rules) or event based (e.g. market will be cancelled due to a high 
price volatility). The rules of a MetaMarket analogously control the coordination of 
the multiple markets it contains. Thus, the concept of MetaMarkets is a promising 
approach to managing the rules for cDMMs and provides the coordination of the 
markets within a cDMM. 
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Start(M 3 ) <= Start(M 4 ) 
and 

Stop(M 3 ) >= Start(M 4 ) 



c) 

Stop(M 5 ) < Start(M 6 ) 
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Start(M 7 ) > Start(M 8 ) 
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Stop(M 7 ) > Start(M 8 ) 



Fig. 4. Combinations of market models. 



As described above, cDMMs are defined by a set of market models and the coordi- 
nation of the market models by rules. In all probability, the most intuitive rule be- 
tween market models is based on the time. Time-based rules dictate, e.g. the starting 
and stopping times of markets, and thus formulate time-based relations between the 
markets. Let Start(Mj) be the starting time of market Mj and Stop(M ; ) the stopping 
time of it and let StartlMj) < Stop(Mj) for i=l,...,8. 

Figure 4 presents, in four cases, different possible combinations of two market 
models with different starting and stopping times. The case a) shows markets Mj and 
M 7 being valid one by one. In case b) market M 3 starts before market M 4 and ends 
after it. In case c) the situation in which Market M 5 stops before M 6 starts: Stop(M 5 ) < 
Start(M 6 ) is focused. The case of overlapping markets is shown in d). 
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The example mentioned can be extended to event-based starting and stopping 
rules. In that case, the starting and stopping times are not fixed but flexible, depending 
on events, that trigger the starting and the stopping of markets. To specify the case of 
event-based rules, consider a market structure such as an English Auction. The Eng- 
lish Auction can be stopped 10 minutes after the last order has been inserted. In this 
case, the stopping time is relative to the event "last order inserted" and thus the stop- 
ping rule of the English Auction is event based. 




Fig. 5. Communication between an order and a MetaMarket. 



MetaMarket is a concept to manage cDMMs especially to coordinate multiple 
market models throughout rules. In addition to this, MetaMarkets also have to “com- 
municate" their internal structure to its environment. Consider the case of an investor 
choosing a cascading dynamic market (cDM) for his order to be traded in (cDM is an 
instance of cDMM). Thus, the order has to register in the cDM and with its registra- 
tion the order receives information about the internal structure of the cDM i.e. the 
sequence of markets and their coordination. Figure 5 (cf. also Figure 2) describes this 
situation. An order O is entering a MetaMarket MM. First, O communicates MM its 
interest to be traded in MM (Step 1). Then MM communicates to O the sequence of 
markets it has to pass through (Step 2). According to this information, O initialises its 
own rules and can be traded in MM. 

5 Implementation of MetaMarkets 

As described above, a MetaMarket is a coherent combination of markets and rules. 
The description of MetaMarkets enables the integration of multiple market models 
and rules between them. This section briefly presents the description of MetaMarkets 
in XML. In the XML-file, the markets belonging together are managed as a single 
unit. Figure 6 presents a draft of an XML-structure for the description of MetaMar- 
kets. 

The area <Relationships> . . . </Relationships> contains informa- 
tion about all relationships between market structures. 

The information between the tags <Relationship> . . . </Relationship> 
describe the relationship of a single market structure with the whole MetaMarket or 
paired relationships between market models. The relationships are defined by events 
(as in time-based instances) that regulate the starting or stopping of concrete markets 
within the MetaMarket. We define the following syntax for the definition of events: 
Market [Market _Name( StartEvent(Entry);StopEvent(Exit) ], whereby specifications 
combined by logical operations for both events are allowed. For example, if the mar- 
ket model “StockMarket” should be activated at 12:00 am and finished at 16:00 pm 
we can specify it as follows: 

Market [StockMarket( clocked( 12:00);clocked( 16:00) ]. 
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<MetaMarket> 




<Relationships> 




<Relationship> . . . </Relationship> 


<Relationship> . . . </Relationship> 


</Relationships> 




<OrderSequenceInf ormation>...</OrderSequenceInf ormation> 


<!- -contains links 


to all markets that are included into this 


MetaMarket - -> 




<MarketList> 




<Market> . . . 


</Market> 


<Market> . . . 


</Market> 


</MarketList> 




</MetaMarket> 





Fig. 6. An XML-draft for a MetaMarket. 



Identification of 
subsequences 



1 


2 


3 


4 


5 


6 


M 


Ml 


Ml 


i n 

.3?..... 


MM2 


MM3 




MM6 




P(MM1, 

MM2) 


MM 


4 




MM7 




P(MM1, 

MM3, 

MM4) 


MM4 


P(MM5, 

MM6, 

MM7) 


P(MM5, 

MM7) 


MM7 



S(P(MM 1 ,MM2), P(MM 1 ,MM3 ,MM4), MM4, P(MM5,MM6,MM7), P(MM5,MM7), MM7) 



Fig. 7. Order sequence information in the MetaMarket. 



The area <OrderSequencelnf ormation> . . . </OrderSequencelnfor- 
mation> contains information that a MetaMarket has to “communicate” to an enter- 
ing order (cf. Figure 6). From the order point of the view, the information submitted 
from the MetaMarket describes any combination of market conceivable in a 
MetaMarket. The principle idea is that any combination of market models is imagin- 
able as a sequence of parallel and sequential market models. Therefore, we use a 
grammar to describe a sequence of parallel (P) and sequential ( S ) market models. 
With this grammar, we are able to define any combination of market models. In the 
concrete implementation of this grammar, additional information are used to describe 
events that are used to trigger the state of the order. 

Figure 7 clarifies the grammar described above. The chart represents the informa- 
tion an order receives at the moment it enters a MetaMarket. In this example, the 
order has to visit seven markets in six subsequences. In the first subsequence, the 
order enters market MM1 and MM2 synchronously. After exiting market MM2, the 
order enters markets MM3, MM4, and so on. As already mentioned, the events that 
trigger entry or exit can be time based, or specifically defined in the order, in the mar- 
ket model, or in the MetaMarket. 
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The definition of the market models used in a MetaMarket is given between the 
tags <MarketList> ... </MarketList>. These tags isolate a group of objects that define 
the concrete market models in the current MetaMarket. 

To keep a single market model independently usable in any MetaMarket, it is not 
physically included into the MetaMarket definition, but linked. This leads to the lay- 
ered architecture of MetaMarkets presented in Figure 8. 

The order layer contains four orders and three MetaMarkets in the MetaMarket 
layer. The market layer contains the available markets - Ml, M2, M3, and M4. 
MetaMarketl contains three markets - Ml, M2, and M3; MetaMarket2 contains mar- 
kets M3 and M4; and MetaMarket3 contains only market M4. 




Fig. 8. Layered Architecture of MetaMarkets. 



MetaMarkets enable the insertion of a single instance of a market into any number 
of MetaMarkets. Consequently, orders that are inserted into different MetaMarkets 
can meet in one market instance common to both of the MetaMarkets. For example, 
in Figure 8 the same instance of M4 is present in both MetaMarket2 and MetaMar- 
kets. Order 3 is inserted into MetaMarket2 and MetaMarket3 and order 4 is inserted 
into MetaMarket3. In this instance, order 3 and order 4 can be traded, although they 
are originally inserted into different MetaMarkets. Additionally, order 3 can be traded 
with order 1, which is inserted into MetaMarketl and MetaMarket3, hence, 
MetaMarketl and MetaMarket2 have a common instance of market model M3. 

6 Concluding Remarks and Future Work 

The main problem in fulfilling the requirements of market participants at the level of 
market modelling is that they are, to some extent, contradictory, and therefore not 
resolvable in conventional electronic trading systems. The concept of MetaMarkets 
offers a practical approach for new kinds of electronic markets. We propose that 
MetaMarkets eliminate some of the problems that arise from the dilemma of liquidity 
versus adaptability because it combines multiple market models in one complex mar- 
ket model, and supports a larger volume of market participants’ requirements. It is fair 
to say that this dilemma does not vanish solely with MetaMarkets. The MetaMarket 
concept provides promising possibilities for market modelling. It transfers the prob- 
lems generated by the dilemma from the level of asking: “What can be done to fulfil 
market participants' requirements?” to the more concrete question: “How can avail- 
able concepts be used to fulfil these requirements?” 
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Further research about market integration with MetaMarkets is needed, as well as 
research about their effects on market quality. Moreover, a rigorous definition of 
economically meaningfully events that trigger the entry or exit of an order into a mar- 
ket is required. After implementation of the MetaMarket concept in the near future, 
more detailed information about its economic effects and usability are needed. As a 
consequence, experiments involving students and simulations are imminent. 
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Abstract. Through computer simulations, this paper evaluates the performance 
of an online multidimensional auction system with negotiation support and es- 
pecially focuses on investigating the efficacy of two design features of online 
multidimensional auction system on its performance: sellers' feedback and post- 
utility scoring method. The performance of the auction system is measured by 
joint gain and speed of convergence. The simulation results demonstrate that the 
use of sellers' feedback and post-utility scoring method lead to better bargaining 
outcomes as measured by the buyer’s total utility and the number of auction 
rounds. The research results provide important theoretical implications about the 
role of information feedback in auction design. 



1 Introduction 

Although auctions have been studied by economists for a long time, only recently has 
the online multidimensional auction mechanism received attention from researchers as 
an efficient way of resolving one-to-many bargaining problems [Bichler 2000], 
Among the early studies on multidimensional auctions [McAfee & McMillan 1988], 
Che [1993] studied design competition in government procurement by developing a 
model of two-dimensional auctions, where firms bid on both price and quality, and 
bids are evaluated by a scoring rule set by a buyer. Branco [1997] further extended 
Che’s model by incorporating the impact of costs’ correlations on the design of multi- 
dimensional auction mechanisms. According to his analysis, when the costs of the 
several bidders are not independent, the buyer has to use a two-stage auction; in the 
first stage the buyer selects one firm, and in the second stage, he or she bargains to 
readjust the level of quality to be provided. Bichler [2000] provided the first experi- 
mental analysis of multidimensional auctions. He showed that the utility scores 
achieved in multidimensional auctions were significantly higher than those in conven- 
tional auctions. Strecker and Seifert [2002] report on a computer-based laboratory 
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experiment in a sole sourcing scenario of a single, indivisible object and investigate 
whether a multi-attribute reverse English and a multi-attribute reverse Vickrey auction 
institution lead to identical outcomes with respect to the buyer’s utility, suppliers’ 
profits and allocational efficiency. The results show no significant difference in sup- 
pliers’ profits. 

This research focuses on investigating the efficacy of two design features of online 
multidimensional auction system on its performance: sellers’ feedback and post-utility 
scoring method. The performance of the auction system is measured by joint gain and 
speed of convergence. The joint gain is operationalized by a utility score achieved by 
the buyer and/or seller [Bichler 2000] and the speed of convergence can be measured 
by the number of rounds taken in the multi-round auction. 

The remainder of this paper is organized as follows. Section 2 explains the use of 
sellers’ feedback and the post utility scoring. Section 3, 4, and 5 proposes the hy- 
potheses, experimental design, and experiment setting respectively, Section 6 gives the 
results of simulation and the final section discusses the conclusions and future research 
issues. 

2 Multidimensional Auction with Seller’s Feedback 
and Post-utility Scoring 

2.1 Use of Sellers’ Feedback 

In the conventional procurement auction, buyers search current market conditions (i.e., 
product feature and price) and create Request for Quotes (RFQs) to initiate the auc- 
tion. Therefore, the buyers’ aspiration levels are determined by the search results. In 
case the buyers fail to properly assess the market conditions when they initiate auc- 
tions (e.g., the buyers’ requirement is set too high for a given budget), the auctions 
lead to failure, causing extra cost and time. Furthermore, the buyers do not receive any 
useful information to adjust their RFQs when the auctions fail. Therefore, the buyers 
must initiate another round without knowing which requirement is set too high within 
a given budget. Such a situation occurs due to incomplete information inherent in the 
auction process. Economists have argued that incomplete information leads to less 
efficient bargaining outcomes [Strecker & Seifert 2003]. 

We argue that this can be resolved by providing the buyers with information on 
market conditions. Our system is designed to foster more efficient outcomes by pro- 
viding sellers’ cost information to buyers. However, exposing detailed cost informa- 
tion can be a sensitive issue for sellers. Therefore, revealing sellers’ cost information 
should be minimized to provide just enough information so that the buyers can initiate 
the next round of the auction with a more reasonable RFQ. 

2.2 Post-utility Scoring Method 

Buyers sometimes do not have perfect knowledge on their utility functions before they 
see the alternatives from sellers. Our system allows buyers to determine their prefer- 
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ence structure after alternatives for all the issues have been determined, that is, after 
collecting offers from all the sellers. Buyers evaluate offers based on their utility func- 
tions, and utility functions are composed of the relative importance of issues and indi- 
vidual utility scores for each issue. 

3 Hypotheses 

We hypothesize that a buyer using MAMENS would achieve better bargaining out- 
comes than those using a conventional multidimensional auction system (Note: 
Throughout this paper, conventional multidimensional auction systems refer to the 
multidimensional auction systems built based on the proposed multidimensional auc- 
tion mechanism but not having the two unique design features of MAMENS). It is also 
hypothesized that an increased number of sellers will lead to better bargaining out- 
comes for the buyers regardless of the trading mechanism. More specifically, the fol- 
lowing formal hypotheses are proposed. 

HI: Using the post-utility scoring method of the MAMENS system will lead to 
more joint gain than not using it. 

H2: An increased number of sellers will lead to more joint gain regardless of utility 
scoring method. 

H3: Providing buyers with sellers’ cost information in the MAMENS system will 
lead to a quicker speed of convergence. 

H4: An increased number of sellers will lead to quicker speed of convergence re- 
gardless of the existence of sellers’ cost information. 

H5: Providing buyers with sellers’ cost information in the MAMENS system will 
lead to more joint gain than not providing it. 

H6: An increased number of sellers will lead to more joint gain regardless of exis- 
tence of the sellers’ cost information. 



4 Experimental Design 

In order to test the hypotheses, two independent experiments were conducted using 
computer simulation as in Bichler [2001 ]. Details of the computational platform used 
for the simulation is presented in the following sections. While holding other variables 
constant, each experiment employed two independent variables: the treatment and 
number of sellers. In each experiment, there were three groups with a different num- 
ber of sellers (3, 5, and 7). The computer simulation ran 30 sessions for each group 
under the two conditions: one with treatment and the control group without treatment. 
Except for the treatment, all the parameter values were the same within the experi- 
ment. 

Experiment I 

Hypotheses 1 and 2 (the effect of the post-utility scoring method) were tested in Ex- 
periment I. A buyer initiated the multidimensional auction with a maximum level of 
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price threshold (i.e., 100) in order to avoid the failure of first round bargaining due to a 
tight budget. Sellers were assigned randomly to the two conditions: MAMENS (with 
the post-utility scoring method) and conventional multidimensional auction system 
(without the post-utility scoring method). To investigate the effect of the post-utility 
scoring method, the mean values of the buyer’s total utility (i.e., joint gain) from both 
conditions were compared using ANOVA test. 

Experiment II 

In Experiment II, hypotheses 3, 4, 5, and 6 (i.e., the effect of sellers’ feedback) were 
tested. In this experiment, unlike in Experiment I, a buyer initiated the multidimen- 
sional auction in each trading session with an unreasonably low level of price thresh- 
old (i.e., 68), purposefully causing the failure of first round bargaining. After the fail- 
ures of each round, the buyer relaxed one requirement per round in order to increase 
the chances of receiving a satisfactory offer in the next round. The auction continued 
to run new rounds until the buyer found a satisfactory offer. 

There were two treatments in Experiment II to illustrate relaxing requirements: 
MAMENS (with sellers’ feedback) and conventional multidimensional auction system 
(without sellers’ feedback). The buyer using MAMENS relaxed the requirement that 
was most cost-causing to the majority of sellers. On the other hand, the buyer using 
the conventional multidimensional auction system relaxed the requirement that was 
personally least important. 

Investigating the effect of sellers’ feedback is more complex than investigating the 
effect of the post-utility scoring method. The number of rounds taken to reach agree- 
ment as well as the mean value of the buyer’s total utility is used as a measure of ef- 
fectiveness of the two systems. The number of rounds would reveal the efficiency of 
the negotiation support capability whereas the buyer’s total utility value would show 
the quality of the final bargaining outcomes. These two dependent variables were 
compared using ANOVA test. 



5 Experiment Setting 

5.1 Task 

The task used in the experiments is digital camera trading. The hypothetical digital 
camera trading game involves two issues in addition to the price: resolution (number 
of pixels) and delivery time (number of days). The total cost of a camera depends on 
only two factors: the cost of resolution and delivery time. Each trading session in- 
volved one computer- simulated buyer and various numbers of computer- simulated 
sellers (3, 7, and 1 1). To investigate the research questions in Experiment I, we run the 
computational platform with two different modes to calculate the overall utility to the 
buyer: MAMENS mode (with post-utility scoring) and conventional auction mode 
(without post-utility scoring). In Experiment I, the winning offers determined by the 
two different modes are compared. In Experiment II, only MAMENS mode is used 
because the experiment’s focus is not the effect of different utility scoring methods. 




130 Sungwon Cho et al. 



5.2 Parameters 

The computational platform involves a number of parameters that are considered to 
affect the bargaining outcomes. In order to focus on testing the research questions, 
some of the parameters are intentionally manipulated or held constant by the re- 
searcher, while others are randomly assigned by the computer (Table 1). 



Table 1. Parameters used in the computer simulation. 



Parameter name 


Parameter value used in the simulation 


Number of sellers 


3,7,11 


Number of issues 


3 


Number of trading session runs 


30 


Buyer’s price threshold 


100 in Experiment 1 , 68 in Experiment II 


Value weight 


54:23:23 (Price: Resolution: Delivery time) 


Cost weight 


Random number between 0.2 and 0.8 



Number of Sellers 

Most previous bargaining experiments used a fixed number of bidders because of the 
cost for human subjects. Bichler [2000] used four sellers in his multidimensional auc- 
tion experiment. We, however, use different numbers of sellers for each experiment in 
order to investigate the effect on the bargaining outcome of a changing the number of 
sellers. For each experiment the computational platform runs the same simulation with 
three different group sizes: 3, 7, and 1 1 respectively. 

Number of Issues 

A multidimensional auction can involve many issues, although price is the main con- 
cern for the buyer. Throughout the simulation the number of issues is being held to 
three - price, delivery time, and resolution - in order to focus specifically on the cur- 
rent research questions. 

Number of Trading Sessions 

For each study, the computational platform runs 30 trading sessions in order to provide 
sufficient data for the statistical analysis. Each session might include several trading 
rounds if the buyer cannot find the winning offer in the first round. Otherwise, one 
session of bargaining will include only one trading round. 

Buyer's Price Threshold 

In the simulation, the buyer has a price threshold or a budget limit that the sellers’ 
offers cannot exceed in order to be accepted. Therefore, it can be assumed that the 
lower the price threshold the less the chance of finding an offer that fulfills the RFQ. 
Different buyers’ price thresholds are used for each experiment. In Experiment I, the 
purpose is to examine the effect of the utility scoring method and therefore subsequent 
bargaining rounds are not performed. Therefore, buyers hold the highest possible 
threshold (100) in order to avoid impasse in the first round. In Experiment II, on the 
other hand, the threshold is set significantly lower (68), in order to make it harder for 
sellers to fulfill the RFQ within the price threshold. The threshold of 68 was chosen 
because it was the value that led to impasse in the first round of bargaining in the pilot 
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testing of the simulation. However, as the buyer keeps relaxing requirements in subse- 
quent rounds, sellers can also reduce the total cost, increasing the chance of fulfilling 
the RFQ within the price threshold. 

Value Weight 

The relative importance of issues can affect the bargaining outcomes of multidimen- 
sional auctions [Bichler 2000]. Therefore, it is important to balance the weight distri- 
butions and keep them constant. The pilot experiments of this study sought to use 
perfectly balanced weight distribution (e.g., 34:33:33). However, it was determined 
that there was a difference between the weight for price and the other two issues. Price 
is a continuous attribute in terms of determining utility score while the other two is- 
sues are discrete attributes. The pilot experiments showed that the weight on the price 
tends to be under-represented compared to the other weights due to the different utility 
scoring method. In other words, if all three issues have the same weight, the impact of 
price change is less than those of the other two. This simulation, therefore, put more 
weight on the price. However, the same weight distribution, (54:23:23), is used 
throughout the entire simulation. 

Cost Weight 

Unlike the buyer who has three value weights, sellers have only two cost weights: one 
for resolution and the other for delivery time. The cost weight is randomly determined 
by the computer within the range of 0.2 and 0.8 adding up to 1. 



6 Simulation Results 

6.1 Experiment I: The Effect of the Post-utility Scoring Method 

In Experiment I, for all groups, the utility scores achieved by the buyer in the post- 
utility selection treatment were significantly above those achieved by the conventional 
method. Using a two-way ANOVA (a=0.05), the null hypothesis of revenue equiva- 
lence between the post-utility scoring method and the conventional method was re- 
jected, and the hypothesis 1 that MAMENS’s post-utility scoring method would 
achieve higher utility scores than the conventional method was supported. Also, the 
number of sellers had a significant effect on the outcome. Therefore, hypothesis 2 was 
also supported. Although there was not a specific hypothesis related to the interaction 
effect, it was investigated using a two-way ANOVA test and no significant interaction 
effect was found. Table 2 shows the results of Experiment I. 



Table 2. The results of Experiment I: Hypotheses 1 and 2 (ANOVA). 



ANOVA test results (hypotheses land 2) 


Source of Variation 


Df 


F 


P-value 


Number of sellers 


2 


3.12 


0.047 


Treatment (post-utility scoring vs. conventional) 


1 


19.37 


< 0.0005 


Interaction 


2 


1.17 


0.312 




132 Sung won Cho et al. 



The buyer’s total utility values achieved in the treatment of the post-utility scoring 
method were, on average, 3.27% in group l(group size 3), 4.86 % in group 2 (size 7), 
and 7.62% in group 3 (size 11) higher than those achieved by the conventional 
method. Figure 1 shows the buyer’s total utility value on average for each group. 




□ Post-utility scoring 
method 

■ Conventional 



Fig. 1. Average buyer’s total utility value. 

6.2 Experiment II: The Effect of Sellers’ Feedback 

In Experiment II, for all groups, the number of rounds taken by the buyer to determine 
the winner in the MAMENS system was significantly smaller than that of the conven- 
tional system. Using a two-way ANOVA test (a = 0.05), hypothesis 3 was supported: 
MAMENS leads to quicker agreement than the conventional system. The effect of the 
number of sellers was also significant. Therefore, hypothesis 4 was also supported. 
The interaction effect was not significant. Table 3 summarizes the results and Figure 2 
shows the average number of rounds taken for each group and treatment. 



Table 3. The results of Experiment II: Hypotheses 3 and 4 (ANOVA). 



ANOVA test results (hypotheses 3 and 4) 


Source of Variation 


df 


F 


P- value 


Number of sellers 


2 


15.41 


< 0.0005 


Treatment (MAMENS vs. conventional system) 


1 


13.25 


< 0.0005 


Interaction 


2 


1.19 


0.306 



4 



2 _ 


“ » 1 


0 - 






# seller =3 


# seller =7 


# seller =1 1 


— ♦ — MAMENS 


2.8 


2.57 


2.33 


— ■ — Conventional sys. 


3.33 


2.8 


2.57 



Fig. 2. The average number of rounds taken for each group and treatment. 



For all the groups, the utility scores achieved by the buyer in the MAMENS system 
were also significantly above those of conventional system. Using an ANOVA test 
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(a = 0.05), the null hypothesis of revenue equivalence between the MAMENS system 
and conventional system was rejected, and hypothesis 5 was supported: MAMENS 
achieved higher utility scores than the conventional system. The effect of the number 
of sellers was also significant. The interaction effect was not significant. The results 
are shown in Table 4 and Figure 3 shows the buyer’s total utility value for each group 
in Experiment II. 



Table 4. The results of Experiment II: Hypotheses 5 and 6 (ANOVA). 



ANOVA test results (hypotheses 5 and 6) 


Source of Variation 


Df 


F 


P- value 


Number of sellers 


2 


3.39 


0.036 


Treatment (MAMENS vs. conventional system) 


1 


17.20 


< 0.0005 


Interaction 


2 


1.24 


0.291 



35 



30 - 


♦ t 








# seller =3 


# seller =7 


# seller =1 1 


—♦—MAMENS 


32.63 


32.7 


33.43 


Conventional 


30.17 


31.77 


31.87 



Fig. 3. Buyer’s total utility value in Experiment II. 



7 Discussions and Conclusion 

The research results provide important theoretical implications about the role of in- 
formation feedback in auction design. Koppius et al. [2000] argue that information 
feedback during an auction might have a significant impact on the performance of the 
auction mechanism and their proposition was made regarding the information feed- 
back on improving a bid from a seller’s perspective like in [Bodendorf et al. 1997; 
David et al. 2002]. Parkes [2002] considers auction design in a setting with costly 
preference elicitation and motivates the role of proxy agents situated between bidders 
and the auction, and maintain partial information about agent preferences and compute 
equilibrium bidding strategies based on the available information. The proxy agents 
can also elicit additional preference information incrementally during an auction. 
Parkes [2002] shows that indirect mechanisms, such as proxied ascending-price auc- 
tions, can achieve better allocative efficiency with less preference elicitation than 
direct mechanisms, such as sealed-bid auctions. 

The simulation results presented in this paper validate the proposition from the 
buyer’s perspective. Revealing information by a party can also tilt the information 
balance of power [Koppius et al. 2000]. Although this paper did not investigate this 
issue in depth, it is suggested that a trusted third party can help the participants main- 
tain a balance of power by regulating the degree of information feedback. 
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The research has several limitations. First, the computer simulation was conducted 
in a controlled environment to examine the effect of a limited number of factors. 
Therefore, the research results may turn out differently in real auction situations where 
various unexamined factors are involved that interact with each other. Second, al- 
though this study simulated realistic bargaining situations, some of the parameters 
were necessarily arbitrary. These limitations are inherent in computer simulation and 
can be overcome by a field study. 

In addition to field experiments, there are several promising areas for future re- 
search. There needs to be further investigation on the impact of other variables on the 
performance of multidimensional auction mechanisms. Although this study investi- 
gates the effect of changing the number of bidders in the analysis of multidimensional 
auctions, there are still more variables that might affect the bargaining outcome in 
these multidimensional auctions. For example, the effect of the number of issues on 
the bargaining outcomes has not been thoroughly studied. Future studies need to look 
at such variables and their interactions by extending the simulation model presented in 
this study. 



References 

1. Bichler, M., An Experimental Analysis of Multi-Attribute Auctions, Decision Support Sys- 
tems 29:249-268, 2000. 

2. Bichler, M., The Future of e-Markets, Cambridge University Press, 2001. 

3. Bodendorf, F., Bui, T., Reinheimer, S., A Software-Agent-Based DSS for Supporting an 
Electronic Air Cargo Market, Proceedings of the International Society for Decision 
Support Systems (ISDSS97), Fausanne, Schweiz, 1997. 

4. Branco, F., The Design of Multidimensional Auctions, RAND Journal of Economics, 
28(11:63-81, 1997. 

5. Che, Y., Design Competition Through Multidimensional Auctions, RAND Journal of Eco- 
nomics, 24(4):668-680, 1993. 

6. David, E., Azoulay-Schwartz, R., and Kraus, S., Protocols and strategies for automated 
multi-attribute auctions. Proceedings of the first international joint conference on 
Autonomous agents and multiagent systems, 77 - 85, 2002 

7. Koppius, O., Kumar, M., Heck, E., Electronic Multidimensional Auctions and the Role of 
Information Feedback, Proceedings of the 8th European Conference on Information Sys- 
tems, Vienna, Austria. 2000. 

8. McAfee, R. and McMillan, J., Multidimensional Incentive Compatibility and Mechanism 
Design, Journal of Economic Theory 46:335-54, 1988. 

9. Parkes, D., Price-Based Information Certificates for Minimal-Revelation Combinatorial 
Auctions, Lecture notes in computer science, 2531:103-122, 2002. 

10. Strecker, S. and Seifert, S., Electronic sourcing with multi-attribute auctions. Proceedings 
of the 37th Hawaii International Conference on System Sciences, 2004 

11. Strecker, S. and Seifert, S., Preference Revelation in Multi- Attribute Bidding Procedures: 
An Experimental Analysis, 14th International Workshop on Database and Expert Systems 
Applications, DEXA‘03, September 1-5, Prague, Czech Republic, pp. 850-854, 2003. 




Electronic Negotiations - 
A Generic Approach with Action Systems 



Juho Makio 1 , Ilka Weber 1 , and Christof Weinhardt 1 

University of Karlsruhe, Information Management and Systems 
Englerstrasse 14, 76131 Karlsruhe, Germany 
{maekioe , weber , weinhardt}@iw . uka . de 



Abstract. This paper proposes a domain-independent generic trading platform 
that provides various auction and negotiation types. Requirements of the platform 
in the "generic” context, e.g. domain-independency, reusability, and flexibility, 
are identified and characterised. We propose two comprehensive concepts pro- 
vided by the generic platform: (i) a basic order structure, and (ii) a basic transac- 
tion process. The generic order presents a domain-independent structure defined 
by multiple attributes. The basic transaction process is modelled at a high level 
of abstraction respectively various auction and negotiation protocols. Consider- 
ing the basic transaction process as an action system leads to a finite sequence of 
states. Hence, the sequence of states is not fixed, and each state can be individu- 
ally parameterised. These characteristics enable an individual configuration of an 
abstract execution model for negotiation processes and thus provides genericity 
in electronic negotiations. 



1 Introduction 

The intention of our research work is to develop and implement the concept of a generic 
platform for electronic markets. The platform is the core, or market server of our generic 
system and supports all facets of electronic markets. Ranging form bilateral negotiations 
like chatting, to auction mechanisms, or even more complex negotiation protocols, the 
generic platform is the basic system that enables the automation of trading and negoti- 
ation processes. The platform provides the infrastructure and all necessary services to 
set up electronic markets for electronic negotiations. Basic functions that are common 
to all electronic markets, and thus provided by the market server, are (1) input functions 
that accept input data from outside the system, (2) storage functions that retain input 
data and retrieve stored data, (3) processing functions that calculate, and manipulate in 
other ways, the input and stored data, and (4) output functions that produce processing 
results for use outside the system. 

According to these aspects, the trading platform, or market server, can be defined as 
a run-time environment for electronic markets. Hence, a market is commonly defined 
as a physical, or virtual, location where price is determined and buy and sell orders are 
matched to create trades according to a set of rules that govern the processing of these 
orders. The definition given in [1 ] states that “electronic markets are based on technol- 
ogy and are highly automated, providing different types of services for investors”. 
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Bakos [2] describes electronic markets as “inter-organizational information systems 
that allow buyers and vendors to exchange information about prices and product offer- 
ings”. Common to these definitions is that an electronic market carries out a market 
with technical aids to fulfil the needs of buyers, sellers and other information carriers 
in respect of information dissemination and exchange. In this context, trading describes 
the interaction and coordination between buyers and sellers to exchange information, 
goods, services, and payments. The interaction and coordination process comprises 
standardised as well as complex transaction processes, e.g. auctions and negotiations. 
However, electronic markets support the transaction processes mentioned above, en- 
abling multiple buyers and sellers to interact, and provide additional services and tools. 
Most traditional electronic markets or negotiation systems provide coordination mech- 
anisms, ergo one pricing mechanism. 

Since all auctions are negotiations, (but not vice versa, cf. [3]) we consider works 
from both fields of research in this section. McAffee and McMillan [4] define an auction 
as “a market institution with an explicit set of rules determining resource allocation and 
prices on the basis of bids from the market participants”. According to Smith [5], an 
institution, together with an economic environment, defines a microeconomic system. 

1 . The institution / defines M the language of the market and g, the communication 

rules for agents as well as the rules governing the communication process (c.f. [5], 
[6]). Besides this, an institution defines allocation rules h to determine the (pro- 
visional or final) allocation of the commodities, and cost imputation rules c, the 
payment to be made by the agents. The individual property rights of each agent 
i are defined by Ii = (Mi,hi,Ci, gt) specifying the space of possible messages 
Mi agent i may sent, agent i’s allocation hi, agent’s payment Cj, and agent’s mes- 
sage exchange rules gt. The collection of all agents’ property rules results in the 
institutions of the microeconomic system I = , . . . , Ay). 

2. An economic environment e = (ei, . . . , ejy) is defined by the characteristics e* of 
economic agents i £ { 1 , . . . , N} (market participants). Agent’s i characteristic e* 
is defined over a K + 1 dimensional commodity space specifying the agent’s util- 
ity function, a technology (endowment), and a commodity endowment. Hence, the 
environment e is defined as a “set of initial circumstances which can not be altered 
by the agents or the institutions within which they interact” [5]. Furthermore, each 
characteristic of an agent is private and not publicly observable. 

Bringing together both the institution / and the microeconomic environment e, a mi- 
croeconomic system S can be defined: S = (e, I). Thus, considering each agent i , the 
microeconomic system S is given by S = (e, I) = (ei, . . . , ejv, Ii, ■ ■ ■ , In)- 

This paper focuses a dynamic view on the phases of the transaction process and its 
activities of the institution I. The proposed action based approach on the transaction 
process leans on the media reference model (MRM) [7]. 

The MRM proposes a comprehensive concept for electronic negotiations contribut- 
ing to a structured and methodological approach in engineering electronic negotiations. 
The MRM analyses the transaction process that is the interaction of agents on a (elec- 
tronic) medium, e.g. a platform or electronic market, and identifies several phases of 
interaction. Based on the MRM, the contribution of this paper is manifold. We set out 
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to: (1) identify requirements of a generic electronic platform, providing various auction 
types, e.g. a generic order structure and the generic transaction process, and (2) propose 
a basic transaction process on a high level of abstraction. Requirements on the generic 
platform towards the order structure and the transaction process are identified in sec- 
tion 2. Hence, the order structure is not into the scope of this paper and thus we briefly 
sketch the idea of the generic order type. Section 3 focus on the aspects of a generic 
platform and present a basic framework for the generic transaction process. The main 
contribution of this paper is the modelling of the basic transaction process with action 
systems (cf. Section 3). Section 4 gives an example of an English auction, e.g. match- 
ing and allocation, modelled by an action system. In Section 5 we briefly summarise 
the main contributions of our research work. 

2 Platform Requirements 

To focus the genericity of the electronic trading system, the aim is to conceptualise 
and implement a basic system that provides various transaction protocols 1 . Concerning 
electronic trading systems, the generic characteristic has various facets: 

1. Order-Structure (Request-Structure): As the electronic trading system is not deter- 
mined for a specific application domain, the transaction object has to be defined by 
a flexible and generic order structure (generic order). A generic order is constructed 
to serve various applications - the domain-independent structure is defined by mul- 
tiple attributes, not limited to a certain number of attributes. For example in a stock 
market, the order is determined by a ISIN-number, the volume, and the price. In an 
automobile market, a car is specified by the colour, the horsepower, the type, the 
driven kilometres, and the price. The number of the attributes is not limited. 

A generic order can be seen as a framework, seeking to capture much of the com- 
plexity of the transaction objects by encompassing several variables, here attributes. 
Note that a framework identifies and structures the relevant variables. Furthermore, 
it also reveals the interactions between the variables. As such, the theory, the frame- 
work is based upon, comprises the variables, their organization, their interactions 
and finally their relationships. Thus, in our context of the generic order, the frame- 
work has to provide the characteristics of (1) the transaction object and (2) the 
market characteristics. To accomplish these prerequisites, the framework provides 
the dynamic definition and specification of the attributes. As a consequence of this, 
the framework or the generic order is a powerful concept enabling a generic appli- 
cation to various domains. 

2. Reusability of the transaction process: The transaction process has to be defined 
and structured as a reusable process. Here, “reusability” means that the basic struc- 
ture of the process can be used for similar transaction processes, such as auction 
(and later, negotiation) processes. Thus, identical or common activities of auctions 
have to be identified, building a basic structure of the transaction process. The basic 
activities and the basic process are defined within the core in a domain independent 
and flexible manner. 

1 In this context, “generic” is defined as something that is applicable to an entire class or group. 




